Компании
<<  Система кондиционирования коммерческого здания (Торгово-развлекательный комплекс) Опыт автоматизации предприятий целлюлозно-бумажной промышленности  >>
GStreamer as multimedia framework in Android: a new alternative
GStreamer as multimedia framework in Android: a new alternative
Context
Context
GStreamer in Android: why
GStreamer in Android: why
GStreamer to handle what
GStreamer to handle what
GStreamer replace OpenCore
GStreamer replace OpenCore
GStreamer replace OpenCore
GStreamer replace OpenCore
Porting GStreamer on Android, what are the difficulties
Porting GStreamer on Android, what are the difficulties
MediaPlayer services
MediaPlayer services
MediaRecorder services
MediaRecorder services
MetadataRetriever services
MetadataRetriever services
uridecobin optimizations
uridecobin optimizations
uridecobin optimizations (2)
uridecobin optimizations (2)
How to cope with Android licensing
How to cope with Android licensing
Improvements done in GStreamer with Collabora Multimedia support
Improvements done in GStreamer with Collabora Multimedia support
U671X-based phones in mass production
U671X-based phones in mass production
Conclusion
Conclusion
Questions and Answers
Questions and Answers
THANK YOU
THANK YOU

Презентация: «GStreamer as multimedia framework in Android: a new alternative». Автор: epatrud. Файл: «GStreamer as multimedia framework in Android: a new alternative.ppt». Размер zip-архива: 2844 КБ.

GStreamer as multimedia framework in Android: a new alternative

содержание презентации «GStreamer as multimedia framework in Android: a new alternative.ppt»
СлайдТекст
1 GStreamer as multimedia framework in Android: a new alternative

GStreamer as multimedia framework in Android: a new alternative

CELF Embedded Linux Conference Europe 2010 Benjamin Gaignard October 27, 2010

2 Context

Context

ST-Ericsson U671X Android platform on ARM9 @ 416 MHz with video, audio and graphics dedicated hardware accelerators. Multimedia hardware capabilities: Video encode and decode Audio encode and decode Graphics Imaging Camera Linux kernel 2.6.29 GStreamer 0.10.26

2

October 27, 2010

3 GStreamer in Android: why

GStreamer in Android: why

GStreamer is a mature multimedia framework. GStreamer is a flexible framework. GStreamer is supported by an active community. ST-Ericsson U671X platform was developed and matured since years using GStreamer as multimedia framework with success. GStreamer is required by most of Linux framework: MEEGO, Ubuntu GStreamer graph based approach is naturally fitting U671X hardware split (1 GStreamer element for 1 HW block)

3

October 27, 2010

4 GStreamer to handle what

GStreamer to handle what

Multimedia playback: audio/video streams from local storage, streaming or progressive download. Multimedia recording: Camera application, Sound Recorder. Metadata retrieving: audio/video tags, bitrates, codec information, thumbnails generation. Constraints: Do not touch to Android java APIs Follow Android versions (cupcake, donut, ?clair …) Respect internal Android APIs

4

October 27, 2010

5 GStreamer replace OpenCore

GStreamer replace OpenCore

5

October 27, 2010

6 GStreamer replace OpenCore

GStreamer replace OpenCore

6

October 27, 2010

7 Porting GStreamer on Android, what are the difficulties

Porting GStreamer on Android, what are the difficulties

Android isn’t built from classical Linux way (no pkg-config, no configure, no makefile…) it is required to adapt GStreamer build process to Android one. Glib isn’t provided natively by Android, need to add it. All of this have required to rewrite +70 Android makefiles (.mk) … but it was also an opportunity to carefully select the embedded elements to optimize GStreamer memory foot print and speed. Today U671X android platform embeds 273 elements split in 39 GStreamer plugins. Only use dynamic libraries to not break GStreamer (LGPL) and Android (Apache) licenses terms.

7

October 27, 2010

8 MediaPlayer services

MediaPlayer services

Playbin2 is used to handle playback, streaming and progressive download services Playbin2 has been customized to reduce memory consumption, with Collabora Multimedia partnership. Use 2 dedicated sinks for audio and video rendering: Audio sink requesting adaptation in Android AudioTrack class. Make match GStreamer states (PAUSED, PLAYING …) and events (EOS, SEEK, ASYNC-DONE) to Android MediaPlayer expected states and messages. All the complexity to handle local playback, streaming or progressive download is hidden by playbin2.

8

October 27, 2010

9 MediaRecorder services

MediaRecorder services

Use dedicated GStreamer pipeline with hardware accelerated plugins. Only a limited number of codec supported in Android: Video: MPEG4, H263, H264 Audio: AMR NB, AMR WB, AAC-LC Only one muxer required to handle all recording formats: gppmux v4l2src isn’t use as video source: Android Camera class provide the video frames.

9

October 27, 2010

10 MetadataRetriever services

MetadataRetriever services

Thumbnail generation for video files uses hardware accelerator for video decoding and color space conversion (YUV2RGB). Android metadata tags are mapped to GStreamer tags to extract: codec, bit rate, album art, ... Remove OpenCore implementation from Android MediaScanner class => everything is done by GStreamer. Use a simpler graph than playbin2 for better performance and without cpu/mem heavy cost. uridecodebin uri=%s ! icbtransform ! appsink caps="video/x-raw-rgb,bpp=16” Only video is decoded (not audio) for thumbnail generation.

10

October 27, 2010

11 uridecobin optimizations

uridecobin optimizations

For thumbnails generation we don’t need to decode audio stream, we use uridecodebin’s “autoplug-continue” callback to limit graph building.

11

October 27, 2010

12 uridecobin optimizations (2)

uridecobin optimizations (2)

For metadata retrieving use the same technique but don’t graph video too. It saves time at graph building, reduce memory usage, don’t use any hardware resource.

12

October 27, 2010

13 How to cope with Android licensing

How to cope with Android licensing

Android is under APACHE license. GStreamer is under LGPL v2.1 license. The wrapper between Android multimedia library and GStreamer is under ST-Ericsson copyright and is dedicated to U671X hardware platform. GStreamer libraries are dynamically linked in Android framework. Add NOTICE files in GStreamer to match with Android build process.

13

October 27, 2010

14 Improvements done in GStreamer with Collabora Multimedia support

Improvements done in GStreamer with Collabora Multimedia support

avidemux and qtdemux parsing speed up and memory consumption reduction. Add “push mode” to avidemux, qtdemux and flvdemux. New RTSP buffering mode to handle data burst on mobile networks. queue2 ring buffer mode improvement. QoS message to detect framedrop. All those improvements (+ trivial bugs correction) have being released under LGPLv2.0 and are now available on GStreamer main stream.

14

October 27, 2010

15 U671X-based phones in mass production

U671X-based phones in mass production

ACER betouch E120

ACER betouch E110

ACER betouch E130

HTC Tianyi

15

Enter date here

16 Conclusion

Conclusion

With minimum effort, GStreamer brings to Android: A full open source multimedia framework. Maturity. Additional codecs (VC1, DIVX…) and demuxers (AVI, FLV…) Community support and reactivity. Performance. Evolution (future codec, new streaming protocols, Video Telephony).

16

October 27, 2010

17 Questions and Answers

Questions and Answers

18 THANK YOU

THANK YOU

«GStreamer as multimedia framework in Android: a new alternative»
http://900igr.net/prezentacija/ekonomika/gstreamer-as-multimedia-framework-in-android-a-new-alternative-246426.html
cсылка на страницу

Компании

40 презентаций о компаниях
Урок

Экономика

125 тем
Слайды
900igr.net > Презентации по экономике > Компании > GStreamer as multimedia framework in Android: a new alternative