Картинки на тему «Video Indexing and Summarization using Combinations of the MPEG-7 Motion Activity Descriptor with other MPEG-7 audio-visual descriptors» |
Тексты на английском | ||
<< The achievements of the scientific school of Professor Yuri Kuznetsov in the machine tool industry | Metaphorical Imagery:- Utopian and Dystopian faces of the everyday >> |
Автор: Anthony Vetro. Чтобы познакомиться с картинкой полного размера, нажмите на её эскиз. Чтобы можно было использовать все картинки для урока английского языка, скачайте бесплатно презентацию «Video Indexing and Summarization using Combinations of the MPEG-7 Motion Activity Descriptor with other MPEG-7 audio-visual descriptors.ppt» со всеми картинками в zip-архиве размером 789 КБ.
Сл | Текст | Сл | Текст |
1 | Video Indexing and Summarization using | 33 | to surges and dips in motion activity |
Combinations of the MPEG-7 Motion Activity | (perceived motion) Thus, for a given | ||
Descriptor with other MPEG-7 audio-visual | sport, we can look for certain temporal | ||
descriptors. Ajay Divakaran MERL - | patterns of motion activity that would | ||
Mitsubishi Electric Research Labs Murray | indicate an interesting event In sports | ||
Hill, NJ. | highlights, the emphasis is on key-events | ||
2 | Outline. Introduction MPEG-7 Standard | and not on key-frames. 33. | |
Motivation for proposed techniques Video | 34 | Motion Activity Curve. Shot Detection | |
Summarization using Motion Activity Audio | not meaningful for our purpose Compute | ||
Assisted Video Summarization Principal | motion activity (avg. mag. Of mv’s) for | ||
Cast Detection with MPEG-7 Audio Features | each P-frame Smooth the values using a 10 | ||
Automatic generation of Sports Highlights | point MA filter followed by a median | ||
Target Applications Personal Video | filter Quantize into binary levels of high | ||
Recorder Demonstration Initial work on | and low motion using threshold Low | ||
Video Mining Conclusion. 2. | threshold for Golf, High for Soccer. 34. | ||
3 | Team. Yours Truly Kadir A. Peker – | 35 | Activity Curves for Golf. 35. |
Colleague and Ex-Doctoral Student | 36 | Activity Curve for Soccer. 36. | |
Regunathan Radhakrishnan – Current | 37 | Highlights extraction : Golf. Play | |
Doctoral Student Romain Cabasson – Summer | consists of long stretches of low activity | ||
Intern Ziyou Xiong – Summer Intern and | interspersed with bursts of interesting | ||
Current Collaborator Padma Akella – | high activity Look for rising edges in the | ||
Initial Demo designer and developer | quantized motion activity curve | ||
Pradubkiat Bouklee – Initial Software | Concatenate ten second segments beginning | ||
developer. 3. | at each of the points of interest marked | ||
4 | MPEG-7 Objectives. To develop a | above The concatenation forms the desired | |
standard to identify and describe the | summary. 37. | ||
multimedia content Formal name: Multimedia | 38 | Highlights Extraction: Soccer. Play | |
Content Description Interface Enable quick | consists of long stretches of high | ||
access to desired content whether local or | activity Interesting events lead to | ||
not. 4. | non-trivial stops in play leading to a | ||
5 | MPEG-7: Key Technologies and Scope. | short stretch of low MA Thus we look for | |
Description consumption. Description | falling edges followed by a non-trivially | ||
Production. 5. | long stretch of low motion activity We are | ||
6 | MPEG-7 and other Standards. Rate. | able to find the interesting events this | |
Functionality. Emphasis on Subjective | way but have many false alarms With our | ||
Representation. Emphasis on Semantic | interface false alarms are easy to skip. | ||
Conveyance. MPEG-2 Studio, DTV. Hybrid | 38. | ||
Content Interactive TV, Video | 39 | Strengths and Limitations of Our | |
Conferencing. Indexing Retrieving | Approach. The extraction is rapid and can | ||
Browsing. MPEG-4 SNHC Object-Based. MPEG-7 | be done in real time We use an adaptively | ||
Descriptors. MPEG-1 H.263. JPEG JPEG-2000. | computed threshold that is suited to the | ||
Visualization. Abstract Representation | content An interface such as ours helps | ||
Virtual Reality. 6. | skip false alarms easily There are too | ||
7 | MPEG-7 framework. MPEG-7 standardizes: | many false alarms. 39. | |
Descriptors (Ds): representations of | 40 | Current Approach to Extraction of | |
features to describe various types of | Soccer Highlights. 40. | ||
features of multimedia information to | 41 | 41. | |
define the syntax and the semantics of | 42 | Summary of Sports Highlights | |
each feature representation Description | Generation. Motion Activity provides a | ||
Schemes (DSs) to specify pre-defined | quick way to generate sports highlights We | ||
structures and semantics of descriptors | use a different strategy with each sport | ||
and their relationship Description | The simplicity of the technique allows | ||
Definition Language (DDL) to allow the | real-time tuning of thresholds to modify | ||
creation of new DSs and, possibly, Ds and | highlights Interactive interfaces enable | ||
to allows the extension and modification | effective use. 42. | ||
of existing DSs – XML MPEG-7 Schema. 7. | 43 | PVR: Personal Video Recorder. With | |
8 | MPEG-7 Motion Activity Descriptor. | Massive Amounts of Locally Stored Content, | |
Feature Extraction from Video Uncompressed | Need to Locate & Customize Content | ||
Domain Color Histograms - Zhang et al | According to User. Local Storage. Feature | ||
Motion Estimation - Kanade et al | Extraction & MPEG-7 Indexing. Video | ||
Compressed Domain DC Images - Yeo et al, | Codec. Browsing & Summarization. | ||
Kobla et al Motion Vector Based - Zhang et | Enhanced User Interface. 43. | ||
al Bit Allocation - Feng et al, Divakaran | 44 | Blind Summarization – A Video Mining | |
et al. 8. | Approach to Video Summarization. Ajay | ||
9 | Motivation for Compressed Domain | Divakaran and Kadir A. Peker Mitsubishi | |
Extraction. Compressed domain feature | Electric Research Laboratories Murray | ||
extraction is fast. Block-matched motion | Hill, NJ. | ||
vectors are sufficient for gross | 45 | Content Mining. What is Data Mining? | |
description. Motion vector based | It is the discovery of patterns and | ||
calculation can be easily normalized | relationships in data. Makes heavy use of | ||
w.r.t. encoding parameters. 9. | statistical learning techniques such as | ||
10 | Motivation for Descriptor. Need to | regression and classification Has been | |
capture “pace” or Intensity of activity | successfully applied to numerical data | ||
For example, draw distinction between | Application to multimedia content is the | ||
“High Action” segments such as chase | next logical step Most applicable to | ||
scenes. “Low Action” segments such as | stored surveillance video and home video | ||
talking heads Emphasize simple extraction | since patterns are not known a priori | ||
and matching Use Gross Motion | Should enable anomalous event detection | ||
Characteristics thus avoiding object | leading to highlight generation Not | ||
segmentation, tracking etc. Compressed | applicable at first glance to consumer | ||
domain extraction is important. 10. | video. 45. | ||
11 | Proposed Motion Activity Descriptor. | 46 | Content Mining vs. Typical Data |
Attributes of Motion Activity Descriptor | Mining. Commonalities Large data sets. | ||
Intensity/Magnitude - 3 bits Spatial | Video is well known to produce huge | ||
Characteristics - 16 bits Temporal | volumes of data Amenable to statistical | ||
Characteristics - 30 bits Directional | analysis – Many of the machine learning | ||
Characteristics - 3 bits. | tools work well with both kinds of data as | ||
12 | MPEG-7 Intensity of Motion Activity. | can be seen in the literature and our | |
Expresses “pace” or Intensity of Action | research as well Differences Number of | ||
Uses scale of 1-5, very low - low - medium | features not necessarily as large as | ||
- high - very high Extracted by suitably | conventional data mining data sets Size of | ||
quantizing variance of motion vector | dataset not necessarily as large as | ||
magnitude Motion Vectors extracted from | conventional data mining data sets Popular | ||
compressed bitstream Successfully tested | data mining techniques such as CART may | ||
with subjectively constructed Ground | not be directly applicable and may need | ||
Truth. 12. | modification In summary, new mining | ||
13 | Video Summarization using Motion | techniques that retain the basic | |
Activity. Video sequence V:{f1, f2, … fN} | philosophy while customizing the details | ||
set of temporally ordered frames Any | will have to be developed. 46. | ||
temporally ordered subset of V is a | 47 | Summarization cast as a Content Mining | |
summary Previous work: Color dominant | Problem. DVD “Auto-Summarization” mode | ||
Cluster frames based on image similarity | inspires “blind Summarization” Content | ||
Select representative frames from | Summarization can be cast as follows: | ||
clusters. 13. | Classify segments into common and uncommon | ||
14 | Motion Activity as Summarizability. | events without necessarily knowing the | |
Hypothesis: Motion activity measures | domain Common patterns – what this video | ||
intensity of motion hence it measures | is about Rare patterns – possibly | ||
change in the video Therefore it indicates | interesting events May help to categorize | ||
Summarizability Test of the Hypothesis | video, detect style... The Summary is then | ||
Examine relationship between Fidelity of | a combination of common and rare events | ||
Summary and motion activity Results show | Can hybridize with domain-dependent | ||
close correlation and motivate novel | techniques. 47. | ||
summarization strategy. 14. | 48 | Data Mining Basics. Associations Time | |
15 | Fidelity of a Summary. 15. | series similarity Sequential patterns | |
16 | Test of Hypothesis. Segment the test | Clustering “How does region A and B | |
sequence into shots Use the first frame of | differ”, “Any anomaly in A”, “What goes | ||
each shot as its Key-Frame (KF) Compute | with item x” Marketing, molecular biology, | ||
the fidelity of each key-frame as | etc. 48. | ||
described Compute the motion activity of | 49 | Associations. A set of items i1..im; a | |
each shot For each MPEG-7 motion activity | set of transactions containing subset of | ||
threshold Identify shots that have the | items; a database of transactions: Rule X | ||
same or lower motion activity Find the | ? Y (X, Y items) : Support s: s% of | ||
percentage p of shots with unacceptable | transactions have X,Y together Confidence | ||
fidelity (>0.2) Plot p vs the MPEG-7 | c: c% of the time buying X implies buying | ||
motion activity thresholds. 16. | Y Improvement: Ratio of P(X,Y) to | ||
17 | Motion Activity as a Measure of | P(X)*P(Y) Find all rules with support, | |
Summarizability. 17. | confidence and improvement larger than | ||
18 | Conclusions from Experiment. The | specified thresholds. Continuous-valued | |
percentage of shots with unacceptable | extension exists. 49. | ||
fidelity grows monotonically with motion | 50 | Some Basic Aspects. Unsupervised | |
activity In other words, as motion | learning Similar to clustering vs. | ||
activity grows, the shots become | classification Estimation of joint | ||
increasingly difficult to summarize Hence, | probability density Find values of | ||
motion activity is a direct indicator of | (i1,i2,…,in) where P(i1, i2,…,in) is high. | ||
summarizability Question: Is the first | 50. | ||
frame the best choice as a key-frame? 18. | 51 | Current Direction. As a starting | |
19 | Optimal Key-Frame Selection Using | point, try to discover the temporal | |
Motion Activity. Summarizability is an | patterns we used in detecting golf | ||
indication of change in the shot The | highlights Then generalize to patterns | ||
cumulative motion activity is therefore an | across multiple features Associations | ||
indication of the cumulative change in the | between changes, e.g. activity level | ||
shot. 19. | change, speaker change, scene change, etc. | ||
20 | Optimal Key-Frame Extraction Using | 51. | |
Motion Activity. 20. | 52 | Previously observed pattern: Extended | |
21 | Comparison with Opt. Fidelity KF. Mot. | segments of very low activity followed by | |
Activity. Ddsh First Frame. Ddsh proposed | a jump in activity. Corresponds to a | ||
KF. Number of Shots. Very Low. 0.0116. | player preparing for a swing, then hitting | ||
0.0080. 25. Low. 0.0197. 0.0110. 133. | the ball and the camera following the | ||
Medium. 0.0406. 0.0316. 73. High. 0.0950. | ball. 52. | ||
0.0576. 28. Very High. Overall avg. | 53 | Time sequence mining. Find all similar | |
0.0430. 0.0216. 21. | sub-sequences in a given time sequence | ||
22 | Optimal Key-Frame Selection Based on | E.g. motion activity of a video sequence | |
Cumulative Motion Activity. 22. | Previous work mostly query of a given | ||
23 | Audio Assisted Video Browsing: | sub-sequence in a larger sequence. 53. | |
Motivation. Baseline MHL visual | 54 | Mining for Temporal Patterns. Given a | |
summarization works well only when | sequence S(i) and window size w, construct | ||
semantic segment boundaries are well | the set of all subsequences of size w: | ||
defined Semantic segment boundaries cannot | S(1:w), S(2:w+1), …, S(N-w+1:N) Find the | ||
be located easily using visual features | cross-distances between each pair and | ||
alone Audio is a rich source of content | cluster Problem: How can we search for | ||
semantics Should use audio features to | similar sub-sequences for different window | ||
locate semantic segment boundaries. 23. | sizes? 54. | ||
24 | Past Work. Principal Cast | 55 | Point Distance Matrix. Let the |
Identification using Audio – Wang et al | distance between two sub-sequences of size | ||
Topic Detection using Speech Recog. – | w be: The distance between two points is: | ||
Hanjalic etc Semantic Scene Segmentation | Then. 55. | ||
using Audio – Sundaram et al Past work has | 56 | Point Distance Matrix. xi-xi+w. | |
emphasized classification of audio into | xj-xj+w. 56. | ||
crisp categories We would like both a | 57 | Advantages of Using Point Distance | |
crisp categorization and a feature vector | Matrix. Search for diagonal lines of low | ||
that allows softer classification | point-distance Not limited to a given | ||
Generalized Sound Recognition Framework – | window size, look for the longest possible | ||
Casey et al Casey’s work provides a rich | diagonal line of low point-distance values | ||
audio-semantic framework for our research. | By allowing non diagonal lines and curves, | ||
24. | we can utilize “Time Warping” Matching of | ||
25 | MPEG-7 Feature Extraction for | sub-sequences of different lengths. 57. | |
Generalized Sound Recognition. 25. | 58 | Multi-resolution Pattern Discovery. | |
26 | Our approach to Principal Cast | Multi-resolution analysis: Smooth and | |
Detection. MPEG-7 Generalized Sound | sub-sample time series (conventional | ||
Recognition. State Duration Histograms. | multiscale, e.g. wavelets) Analysis with | ||
Our Enhancement. Principal Cast. 26. | various window sizes, matching across | ||
27 | Proposed Audio-Assisted Video Browsing | different window sizes (our method | |
Framework. 27. | automatically handles this). 58. | ||
28 | Audio-Assisted Video Browsing | 59 | Illustration: Segmenting Haiden Video. |
Framework. 28. | Repeating temporal patterns. 59. | ||
29 | MHL application of Casey’s approach to | 60 | Other Issues. Clustering segments |
News Video Browsing. Classify the audio | after finding similarities Extend to other | ||
segments of the news video into speech and | features, multiple dimensions Currently | ||
non-speech categories in first pass | using motion activity only Extend to | ||
Classify the speech segments into male and | multi-dimensional feature vectors (e.g. | ||
female speech Using K-means clustering | color histogram) Extend to multiple | ||
find the “principal” speakers in each | features, multiple modalities (e.g. video | ||
category The occurrence of each of the | + audio) Using a normalized Euclidean | ||
principal speakers provides a natural | distance measure Normalization based on | ||
semantic boundary Apply baseline visual | local variance of data. 60. | ||
summarization technique to semantic | 61 | Block-diagram of time-series mining. | |
segments obtained above There is thus a | 61. | ||
two-level summarization of the news video. | 62 | Target Applications. Surveillance | |
29. | Video Can detect unusual events through | ||
30 | Clustering Results for Male Principal | video mining in stored video Home Video | |
Cast. 30. | Can use event detection and other pattern | ||
31 | Results and Challenges. Moderate | discovery to manage home video | |
accuracy so far. Results are thus | Entertainment Quality Video Blind | ||
promising but not satisfactory Lack of | Summarization Genre Independent yet | ||
noise robustness and content dependence of | event-aware processing Content Management | ||
training process represent major hurdle | for Large Video Databases All of the above | ||
Currently working on eliminating such | at a very large scale. 62. | ||
problems through extensive training | 63 | Future Extension - Model Based | |
Feature extraction too complex – currently | Matching. Use more sophisticated | ||
investigating compressed domain audio | statistical techniques to fuse label | ||
feature extraction Also examining | streams. 63. | ||
alternative architectures that preserve | 64 | Conclusion. System Features Unique, | |
basic spirit of framework. 31. | simple and flexible summarization | ||
32 | Automatic Extraction of Sports | Integrated Player-Browser Enable rapid and | |
Highlights. Rapid Sports Highlights | convenient browsing Video Summarization | ||
extraction is critical Past work has made | using Motion Activity as Summarizability | ||
use of color, camera motion etc. MPEG-7 | Audio-based principal cast detection | ||
Motion Activity Descriptor is simple Can | Audio-visual feature based sports | ||
use it to extract high action segments for | highlights extraction Further | ||
example Should be useful in highlight | Possibilities Refine Audio-assisted | ||
extraction. 32. | browsing Incorporate other visual features | ||
33 | Essential Strategy. Sports are | Video Mining. 64. | |
governed by a set of rules Key events lead | |||
Video Indexing and Summarization using Combinations of the MPEG-7 Motion Activity Descriptor with other MPEG-7 audio-visual descriptors.ppt |
«The english-speaking countries» - The English-speaking countries. Scotland. Great Britain. Australia. Disneyland. USA.
«The animals» - POLAR BEAR. The animals which live in the polar regions. SEAL. GRIFFIN. PANDA. SQUIRREL. KANGAROO. DOLPHIN. LIZARD. FISH. TIGER. The animals which live in the desert. The animals which live in Australia. PARROT. The animals which live in a SAVANNA. BEAR. FLAMINGO. SEA-HORSE. The animals which live in the OCEAN.
«The green movement» - Their features. One of the largest victories гринписовцев in the given campaign can name refusal of flooding of an oil platform brent spar as it contained many toxic substances. The main objective — to achieve the decision of global environmental problems, including by attraction to them of attention of the public and the authorities.
«Переменные Visual Basic» - Присваивание переменным значений. Объявление переменных. Пример программного кода Visual Basic. Byte, short, integer, long, single, double – типы числовых значений. Переменная. Переменные: тип, имя, значение. Имена переменных. A = 216 b = -31576 c = 3.1415926 D = “visual basic” А = А - 10. Типы переменных.
«Женщина the woman» - Оценочная структура лексической единицы “женщина”. As great a pity to see a woman cry as a goose go barefoot. Наименование молодой девушки в современном английском языке. «Un homme»- франц. « A man »- англ. Значение понятия «женщина» в семье. Баба слезами беде помогает. A good wife makes a good husband.
«Healthy life» - Experts say just 30 minutes of activity three to four days per week will help you stay healthier. Exercise. A well balanced diet coupled with regular exercise is a successful combination to help keep your body fit. Bad habits. Here a few: Park father away from work, the grocery the shopping mall Take your dog for a 20-30 minute walk every other day.