extracting frames from movies on iPhone

February 10, 2010

The iPhone SDK provides a high level movie player, but so far there is no way to decode movies or extract frames. In this post I will show you how to do that using FFmpeg.

First, thanks to the FFmpeg project, Martin Böhme for his tutorial on using libavformat and libavcodec and David Conrad for the gas-preprocessor project that allows us to compile FFmpeg for the iPhone.

Before you link the FFmpeg libraries to your project, make sure that you understand FFmpeg’s licensing. My project uses the LGPL version of FFmpeg, which means that I have to make FFmpeg’s source code available (included in the project).

Before you create a commercial project that decodes H264 video, make sure that you get a license from MPEG LA.

The source code is posted on github: git://github.com/lajos/iFrameExtractor.git.

To compile and run the project on your iPhone:

  1. open Terminal
  2. clone the repository: git clone git://github.com/lajos/iFrameExtractor.git
  3. go to the ffmpeg folder in the project: cd iFrameExtractor/ffmpeg
  4. build the ffmpeg libraries: ./build_universal
  5. open the xcode project and run it on your iPhone device

The build_universal shell script creates libraries that work on both armv6 and armv7 processors. If you need libraries for pre-3Gs devices (iPhone 2g, iPhone3G, iPod Touch), use the build_armv6 script. For iPhone 3Gs only libraries, use build_armv7.

iFrameExtractorAppDelegate creates an instance of VideoFrameExtractor (the class that grabs images from the video). When you press the play button, iFrameExtractorAppDelegate sets up a loop and displays all the images from the video until it reaches the end of the movie.

The VideoFrameExtractor class is based on Martin Böhme’s tutorial, upgraded to use swscale. Initialize this class with the full path of the movie:

VideoFrameExtractor *video = [[VideoFrameExtractor alloc]
               initWithVideo:@"/full/path/to/movie.mov"];

By default, the output size will match the movie’s size. You can set outputWidth and outputHeight to change the dimensions:

video.outputWidth = 426;
video.outputHeight = 320;

Use stepFrame to grab the next frame. This method returns false if there are no more frames (reached the end of video):

BOOL result = ;

The processed frame can be accessed as a UIImage on the currentImage property:

UIImage *myImage = video.currentImage;

You can seek to a specific time (in seconds) using the seekTime: method:

;

The movie’s length can be queried by checking the duration property:

NSLog(@"video duration: %f",video.duration);

This should get you started with extracting frames from iPhone movies 😉 If you find any problems, please submit a patch.

Known issues:

  • Images are always decoded as landscape.

55 Responses to “extracting frames from movies on iPhone”

  1. Hi lajos,

    I am also having same problem as “ghotetz”, as follows:

    ld: library not found for -lavformat
    Command /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2 failed with exit code 1

    ld: library not found for -lavformat
    Command /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2 failed with exit code 1

    Can you share its solution with me?

    Thanks in advance.

  2. Hi lajos,

    finally I was able to compile it successfully. it was my silly mistake, without looking into deep I asked you.
    Thanks for your tutorial.

    Keep yourself happy :),
    Sanniv.

  3. Thank you for the nice ‘drop in/starting point sample’ to be able to experiment further with ffmpeg (and on iOS).

    Those trying to build with the (current) iOS 4.1 gm seed (Xcode 3.2.4) SDK may notice that they only have the 4.1 and 3.2 SDK installed (in /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/) and not 4.0 as the build scripts reference.
    To work around this replace occurrences of ‘4.0’ with ‘4.1’ in build_armv6, build_armv7 (and in build_i386 too wouldn’t hurt, even though the iPhoneSimulator4.0.sdk is still available).
    To build (the *libraries*, more changes to the Xcode project might be needed) for the iPad, ‘4.0’ should be replaced with ‘3.2’.

    @Ramkumar: You might find “Session 407 – Editing Media with AV Foundation” from WWDC 2010 interesting/useful. It is available for download through iTunes U. http://developer.apple.com/videos/wwdc/2010/

  4. hi,I have the same problem

    ld: library not found for -lavformat
    Command /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2 failed with exit code 1

    ld: library not found for -lavformat
    Command /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2 failed with exit code 1

    Thanks.
    please help

  5. Hi, I done for configuration and compile the App with ffmpeg lib, but some thing happened on putting my iphone device…

    I got this message :

    dyld: Library not loaded: /usr/local/lib/libavcodec.dylib
    Referenced from: /var/mobile/Applications/66E93E8A-AA25-4D9A-B69F-F4F040D84E17/MyAPP.app/MyAPP
    Reason: image not found

    Would you have any idea ?

    Thanks for watching~

Leave a Reply