Page 1 of 2 12 LastLast
Results 1 to 10 of 18

Thread: HW video decoding

  1. [translate]    #1
    Senior Member
    Join Date
    Apr 2010
    Location
    Austria
    Posts
    769

    Default HW video decoding

    I just opened this thread for funman to post his info ...

  2. [translate]    #2
    Junior Member
    Join Date
    Oct 2010
    Location
    France
    Posts
    22

    Default

    Work in progress

    Patch for frameworks/base : some debug + look for SEC codecs
    Patch for external/opencore : build HW en/decoders (you will need to copy extern_libs_v2/sec/lib/*.so manually in /system/lib)

    status: GetpOMX_GetHandle() fails, and I couldn't locate this function (a function pointer is stored in OMXInterface)


    I'm going to post some information about video decoding, as on 2.2 builds video decoding is either:
    • broken (video do not play)
    • choppy/stuttering (video plays too slowly)


    On Android, video and audio decoding is based on OpenMAX which permits both software and hardware decoding.

    The software decoders are made by PacketVideo, but they are likely much too slow so the CPU is not able to do real time decoding (depending on the codec, the resolution, the audio codec, the bitrates, etc).

    The hardware decoders used the DSP (Digital Signal Processor) present on the S3C6410 SoC (System on Chip) to offload some work from the CPU to the DSP, which is much faster to perform some decoding-related tasks.

    An hardware implementation of audio/video encoders/decoders for the OMAP3 SoC is present in hardware/ti/omap3 (assuming you cloned criminal's repository), but it is likely useless for us since the S3C6410 SoC on the I5700 doesn't have the TI C64x DSP like the OMAP3.

    Software H.264/AVC decoder : in external/opencore/codecs_v2/video/avc_h264/dec/src
    It is also present in frameworks/base/media/libstagefright/codecs/avc/dec/src
    The code is nearly identical, it was copied in frameworks/base/media/libstagefright on 10 december 2009.

    I believe the copy in frameworks/base/media is the one currently being used.

    In frameworks/base/media/libstagefright/OMXCodec.cpp there's a list of decoders (kDecoderInfo) to probe when playing a file: the order is important as the first decoder found will be used, so hardware decoders have higher priority. Extract:
    Code:
        { MEDIA_MIMETYPE_VIDEO_AVC, "OMX.qcom.video.decoder.avc" },
        { MEDIA_MIMETYPE_VIDEO_AVC, "OMX.TI.Video.Decoder" },
        { MEDIA_MIMETYPE_VIDEO_AVC, "AVCDecoder" },
    (This list is present in the /system/lib/libstagefright.so binary)

    For VIDEO_AVC type (AVC == Advanced Video Codec == H.2.64), we'll try qcom decoder (probably Qualcomm), then TI.Video.Decoder (the Texas Instruments decoder for OMAP3, although it's unlikely to run on our phone), and then the (very slow) software decoder.

    Here is what logcat gives me when playing a video file (my build has debug information enabled for libstagefright: I uncommented "#define LOG_NDEBUG 0" in some .cpp files)
    Code:
    V/OMXCodec( 1927): Attempting to allocate OMX node 'OMX.qcom.video.decoder.avc'
    V/OMXCodec( 1927): Attempting to allocate OMX node 'OMX.TI.Video.Decoder'
    V/OMXCodec( 1927): Successfully allocated software codec 'AVCDecoder'
    My first idea was to get this TI.Video.Decoder running but after realizing that the I5700 is not from the OMAP3 family I gave up.


    I also tested a 2.1 build from the kitchen, and H264 playback was perfectly smooth (CPU usage for mediaserver binary was around 20%, instead of 65/80% on 2.2).


    First thing I saw was that on 2.1 system, there are additional libraries: /system/lib/libsXXXYomxoc.so, with
    • XXX = codec (263, 264, aac, ac3, amr, mp4v, vc1, wma)
    • Y = e (encoder) or d (decoder)


    I tried removing this libs264domxoc.so , and the video player couldn't play my H264 file anymore; so the hardware decoder is likely located in this file.


    Now where does it come from? grep told me that it is referenced by libomx_sharedlibrary.so:

    Code:
    $ strings libomx_sharedlibrary.so|grep -E OMX\\. -A 2
    OMX.PV.aacenc
    audio_encoder.aac
    libomx_aacenc_sharedlibrary.so
    OMX.PV.avcenc
    video_encoder.avc
    libomx_avcenc_sharedlibrary.so
    OMX.PV.h263enc
    video_encoder.h263
    libomx_m4venc_sharedlibrary.so
    OMX.PV.mpeg4enc
    video_encoder.mpeg4
    OMX.PV.amrencnb
    audio_encoder.amrnb
    libomx_amrenc_sharedlibrary.so
    OMX.PV.wmadec
    libomx_wmadec_sharedlibrary.so
    OMX.PV.mp3dec
    libomx_mp3dec_sharedlibrary.so
    OMX.PV.amrdec
    libomx_amrdec_sharedlibrary.so
    OMX.PV.aacdec
    libomx_aacdec_sharedlibrary.so
    OMX.PV.wmvdec
    libomx_wmvdec_sharedlibrary.so
    OMX.PV.avcdec
    libomx_avcdec_sharedlibrary.so
    OMX.PV.h263dec
    libomx_m4vdec_sharedlibrary.so
    OMX.PV.mpeg4dec
    OMX.SEC.amrenc
    libsamreomxoc.so
    OMX.SEC.ac3dec
    libsac3domxoc.so
    OMX.SEC.wmadec
    libswmadomxoc.so
    OMX.SEC.aacdec
    libsaacdomxoc.so
    OMX.SEC.amrdec
    libsamrdomxoc.so
    OMX.SEC.h263enc
    libs263eomxoc.so
    OMX.SEC.avcenc
    libs264eomxoc.so
    OMX.SEC.mpeg4enc
    libsmp4veomxoc.so
    OMX.SEC.wmvdec
    libsvc1domxoc.so
    OMX.SEC.h263dec
    libs263domxoc.so
    OMX.SEC.avcdec
    libs264domxoc.so
    OMX.SEC.mpeg4dec
    libsmp4vdomxoc.so
    grep then points to external/opencore/codecs_v2/omx/omx_common/src/pv_omxregistry.cpp and various functions AacRegister(), Mpeg4Register(), AvcRegister() .. which all do:
    Code:
    pCRT->ComponentName = "OMX.PV.mpeg4dec";
    ...
    #if USE_DYNAMIC_LOAD_OMX_COMPONENTS
    ...
    pCRT->SharedLibraryName = "libomx_m4vdec_sharedlibrary.so";
    #endif

    Supposing the order of the strings in the binary is the same than in source code, "OMX.SEC.avcdec" is the name of the hardware decoder present in libs264domxoc.so


    Finally, Leshak just pointed me to a commit which backports Samsung modifications, and these "libs264domxoc.so" and OMX.SEC.* are present, so with this commit video decoding might "just work" already.


    In this case I hope this post will at least help someone to understand how video decoding works in a nutshell.
    Last edited by funman; 11-22-2010 at 06:14 PM. Reason: Add patches

  3. [translate]    #3
    Member
    Join Date
    Jun 2010
    Location
    Poland
    Posts
    66

    Default

    funman, Did you saw this thread? There is a link to some source of opencore codecs. I wonder if anyone tried to compile it in our froyo.

  4. [translate]    #4
    Junior Member
    Join Date
    Oct 2010
    Location
    France
    Posts
    22

    Default

    Thanks; I hadn't seen it.

    From the top of the logs I see addition of Real codecs

    With a quick look it doesn't seem to contain HW decoders, only the software ones.

    EDIT: Yes it is HW codec[COLOR="Silver"]

    ---------- Post added 11-22-2010 at 01:11 AM ---------- Previous post was 11-21-2010 at 10:53 PM ----------

    Patches added in 2nd post
    Last edited by funman; 11-22-2010 at 06:13 PM.

  5. [translate]    #5
    Senior Member
    Join Date
    Feb 2010
    Location
    Singapore
    Posts
    890

    Default

    I've compiled latest xvid codec (Xvid.org: Downloads) for armv6 and here's the resultant library "libxvidcore.so" in zip format --> libxvidcore.zip

    But it's practically useless as it needs a front-end app to be written to specifically use the library, and afaik none exists for android.

    Is there any way to integrate this libxvidcore.so directly into our mediaplayer framework so that it is used by default for xvid format video?


  6. [translate]    #6
    Junior Member
    Join Date
    Oct 2010
    Location
    France
    Posts
    22

    Default

    Quote Originally Posted by ch33kybutt View Post
    I've compiled latest xvid codec (Xvid.org: Downloads) for armv6 and here's the resultant library "libxvidcore.so" in zip format --> libxvidcore.zip

    But it's practically useless as it needs a front-end app to be written to specifically use the library, and afaik none exists for android.

    Is there any way to integrate this libxvidcore.so directly into our mediaplayer framework so that it is used by default for xvid format video?
    xvid doesn't have ARM optimizations so I doubt it will be more useful than the existing m4v decoder by PacketVideo.

    The current software decoder is in frameworks/base/media/libstagefright/codecs/m4v_h263/dec , if you really want you could replace the code with xvid code and modify M4vH263Decoder.cpp accordingly

  7. [translate]    #7
    Member
    Join Date
    Jun 2010
    Location
    china
    Posts
    83

    Default

    I've enable avc/mp4 hardware decode now. one problem is can not make mpeg-4 and mp4-avc(h264) file play together. decoder is no problem, but the native PV_Player and StagefrightPlayer each can only support playing one type of them.

  8. [translate]    #8
    Junior Member
    Join Date
    Oct 2010
    Location
    France
    Posts
    22

    Default

    Do you have a patch or a git repository?

    I think 2.1 supports both mpeg-4 ASP and H264 in hardware.

    By the way I think PV_Player is software decoder (PV = PacketVideo)

  9. [translate]    #9
    Senior Member
    Join Date
    Feb 2010
    Location
    Russia
    Posts
    540

    Default

    Quote Originally Posted by funman View Post
    Do you have a patch or a git repository?
    afaik, it's https://github.com/vflashbirdv

  10. [translate]    #10
    Member
    Join Date
    Jun 2010
    Location
    china
    Posts
    83

    Default

    I registed github but not used. I'll publish my patch and share my source in sevral days. In AOSP,all media codec are software based, manufactures can add their hardware codec using any method they want, modify PV or StageFright source, or make new component. the leshak patch from samsung is modified the PV,but it's older for 2.1 or even 1.6. I partly ported it to 2.2, can play h263 h264 mp4 now, without DivX and has some bugs atm.

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •