ISO/IEC TR 15938-8:2002/Amd 1:2004

Information technology — Multimedia content description interface — Part 8: Extraction and use of MPEG-7 descriptions — Amendment 1: Extensions of extraction and use of MPEG-7 descriptions

Technologies de l'information — Interface de description du contenu multimédia — Partie 8: Extraction et utilisation des descriptions MPEG-7 — Amendement 1: Extensions d'extraction et utilisation des descriptions MPEG-7

General Information

• Status: Published
• Publication Date: 28-Nov-2004
• ICS: 35.040 - Information coding
• ICS: 35.040.40 - Coding of audio, video, multimedia and hypermedia information
• Technical Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Drafting Committee: ISO/IEC JTC 1/SC 29 - Coding of audio, picture, multimedia and hypermedia information
• Current Stage: 6060 - International Standard published
• Start Date: 29-Nov-2004
• Due Date: 31-Jul-2006
• Completion Date: 31-Jul-2006

Relations

• Amends: ISO/IEC TR 15938-8:2002 - Information technology — Multimedia content description interface — Part 8: Extraction and use of MPEG-7 descriptions
• Effective Date: 06-Jun-2022
• Child: ISO/IEC TR 15938-8:2002 - Information technology — Multimedia content description interface — Part 8: Extraction and use of MPEG-7 descriptions
• Effective Date: 15-Apr-2008

Standards Content (Sample)

TECHNICAL ISO/IEC
REPORT TR
15938-8
First edition
2002-12-15
AMENDMENT 1
2004-11-15
Information technology — Multimedia
content description interface —
Part 8:
Extraction and use of MPEG-7
descriptions
AMENDMENT 1: Extensions of extraction
and use of MPEG-7 descriptions
Technologies de l'information — Interface de description du contenu
multimédia —
Partie 8: Extraction et utilisation des descriptions MPEG-7
AMENDEMENT 1: Extensions d'extraction et utilisation des descriptions
MPEG-7
Reference number
ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
©
ISO/IEC 2004
ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
PDF disclaimer
This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but
shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In
downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat
accepts no liability in this area.
Adobe is a trademark of Adobe Systems Incorporated.
Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation
parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In
the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below.

©  ISO/IEC 2004
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means,
electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or
ISO's member body in the country of the requester.
ISO copyright office
Case postale 56 • CH-1211 Geneva 20
Tel. + 41 22 749 01 11
Fax + 41 22 749 09 47
E-mail [e-mail]
Web www.iso.org
Published in Switzerland
ii © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
Foreword
ISO (the International Organization for Standardization) and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide standardization. National bodies that are members of
ISO or IEC participate in the development of International Standards through technical committees
established by the respective organization to deal with particular fields of technical activity. ISO and IEC
technical committees collaborate in fields of mutual interest. Other international organizations, governmental
and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2.
The main task of the joint technical committee is to prepare International Standards. Draft International
Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as
an International Standard requires approval by at least 75 % of the national bodies casting a vote.
In exceptional circumstances, the joint technical committee may propose the publication of a Technical Report
of one of the following types:
 type 1, when the required support cannot be obtained for the publication of an International Standard,
despite repeated efforts;
 type 2, when the subject is still under technical development or where for any other reason there is the
future but not immediate possibility of an agreement on an International Standard;
 type 3, when the joint technical committee has collected data of a different kind from that which is
normally published as an International Standard (“state of the art”, for example).
Technical Reports of types 1 and 2 are subject to review within three years of publication, to decide whether
they can be transformed into International Standards. Technical Reports of type 3 do not necessarily have to
be reviewed until the data they provide are considered to be no longer valid or useful.
Attention is drawn to the possibility that some of the elements of this document may be the subject of patent
rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights.
Amendment 1 to ISO/IEC TR 15938-8:2002 was prepared by Joint Technical Committee ISO/IEC JTC 1,
Information technology, Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia
information.
NOTE This document preserves the sectioning of ISO/IEC TR 15938-8:2002. The text and figures given in this
document are currently being considered as additions and/or modifications to those corresponding sections in
ISO/IEC TR 15938-8:2002.
© ISO/IEC 2004 – All rights reserved iii

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)

Information technology — Multimedia content description
interface —
Part 8:
Extraction and use of MPEG-7 descriptions
AMENDMENT 1: Extensions of extraction and use of MPEG-7
descriptions
Add after subclause 5.6:
5.7 GofGopFeature
This datatype is used to describe a certain visual feature representative of a series of video frames or
collection of pictures. It is obtained by aggregating the visual descriptors extracted from each video frame or
image in the collection.
5.7.1 Feature Extraction
First, the extraction algorithm computes a descriptor of the visual feature for each frame in the sequence or
each image in the collection. The extraction is specified in the subclauses corresponding to the descriptor
used (e.g. for HomogeneousTexture, subclause 4.3.1.1 is used). Once the values of the frame/image-based
descriptors are computed, a instance of GofGopFeature is derived by the aggregation procedure
corresponding to the descriptor used; as defined in ISO/IEC 15938-3.
There are three aggregation methods (i.e. Average, Median, SplitMerge) as follows:
� Average:
Each component of descriptors in the GOF or GOP is summed and then averaged to compose the
aggregated description
� Median:
Each component of descriptors in the GOF or GOP is sorted and then the middle value is selected to
compose the aggregated description.
� SplitMerge:
The DominantColor descriptors from different images are aggregated by merging of the clusters
(“Value” elements) of different descriptors based on their proximity in colour space (the clusters within
the same descriptor are also included as a special case, although if the extraction algorithm from
4.2.3.1 is followed, their distance will be greater than DISTANCE_MIN specified below). The merging
procedure is performed iteratively, starting with the closest pair and repeating until only a small number
of combined clusters remains. The outline of this algorithm is as follows:
closest_distance=0
© ISO/IEC 2004 – All rights reserved 1

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
While (number_of_clusters > MAX_NUM_OF_CLUSTERS or
closest_distance < DISTANCE_MIN) {
1. find two closest clusters
2. merge these two clusters
}
The distance between clusters is defined as the Euclidean distance between cluster centres,
DISTANCE_MIN is the same as in 4.2.3.1 and MAX_NUM_OF_CLUSTERS is equal to 8.
Merging of the clusters is performed as follows. The representative colour value for the merged cluster
is a weighted average of the colour values of the component clusters, where the weights are the
relative pixel counts in the clusters.
m = w m +w m
1 1 2 2
Merging of the colour variances is based on the assumption that each colour component is
independent and for each component we assume that we are calculating the variance of a weighted
sum of two Gaussian distributions. This leads to the following formula for the variance of the merged
cluster σ :
2 2 2
σ = w σ +w σ +w w()m −m ,
1 1 2 2 1 2 1 2
2 2
where σ ,σ are the variances of the component clusters, m ,m are their means and w ,w are
1 2 1 2 1 2
w = W1/(W1+W2), w = W2/(W1+W2)
1 2
where W1 and W2 are the unquantised weights for sub-descriptors.
5.7.2 Similarity Matching Criteria
Matching of GofGopFeature is performed using the descriptors’ matching function appropriate to the
descriptor used. Only GofGopFeature descriptors characterizing the same feature can be compared. For
example, GofGopFeature using the HomogeneousTexture descriptor for two different sequences can be
compared. Some descriptors allow multiple aggregation methods, for example, the Color Layout or Edge
Histogram descriptors. Matching of GofGopFeature describing the same feature but derived with a different
aggregation method is possible.
5.7.3 DDL instantiation examples
In the following two examples, an instance of ColorLayout is embedded in the GofGopFeature datatype.
In the first example, there is no specification of aggregation method.


48
34
32
12 10 13 9 10
14 15
16 12


2 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
In the second example, “Average” is used to aggregate descriptions.


48
34
32
15 11 13 9 8
14 15
16 12


In the following example, an instance of DominantColor is embedded in the GofGopFeature datatype.


0

5
0 89 203
0 1 1


14
120 43 74
0 1 0


12
243 212 27
1 0 0



In the following two examples, an instance of EdgeHistogram is embedded in the GofGopFeature datatype.
In the first example, there is no specification of aggregation method



2 6 4 4 2 1 7 5 3 2 1 6 4 2 2 2 5 4
5 3 1 5 5 6 5 2 6 5 4 4 1 6 4 4 4 0 6 3 5
2 1 5 5 6 6 4 2 3 6 7 3 2 5 5 7 3 2 4 4 7
1 5 6 4 6 1 5 7 4 5 1 6 4 6 5 1 3 4 7 6



In the second example, “Average” is used to aggregate descriptions.
© ISO/IEC 2004 – All rights reserved 3

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)



2 6 4 4 2 1 7 5 3 2 1 6 4 2 2 2 5 4 5 3 1
5 5 6 5 2 6 5 4 4 1 6 4 4 4 0 6 3 5 2 1 5
5 6 6 4 2 3 6 7 3 2 5 5 7 3 2 4 4 7 1 5 6
4 6 1 5 7 4 5 1 6 4 6 5 1 3 4 7 6



In the following two examples, an instance of HomogeneousTexture is embedded in the GofGopFeature
datatype.
In the first example, there is no specification of aggregation method.


19
20

103 87 99 130 97 73 112 109 122 132 108 102 105 113
106 141 103 111 78 76 82 117 88 70 69 61 48 68 48


106 84 94 130 94 75 107 104 117 128 100 99 97 107 92
132 90 106 76 64 78 110 83 65 64 52 39 72 35 47



In the second example, “Median” is used to aggregate descriptions.


19
20

103 87 99 130 97 73 112 109 122 132 108 102 105 113
106 141 103 111 78 76 82 117 88 70 69 61 48 68 48


106 84 94 130 94 75 107 104 117 128 100 99 97 107 92
132 90 106 76 64 78 110 83 65 64 52 39 72 35 47



5.7.3 Conditions of Usage
There are no specific conditions and limitations on the use of this container datatype.

4 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
Add after subclause 6.8:
6.9 Color Temperature
The color temperature of an image specifies the color of illumination in the scene of the image. It is expressed
by Kelvin (K) temperature scale in the [1667K, 25000K] range. Using this, the color temperature descriptor
describes the perceptual temperature feeling of an image. It targets the perception-based image browsing that
enables viewers to navigate and match images based on the temperature perception (i.e. hot, warm,
moderate, and cool) of the image.
This descriptor is also useful when a user would like to change the illumination of scene (i.e. still images or
video) in favor of the user’s preference. For example, some people might want to see warmer images (e.g.
taken under incandescent lights) than original images while some people might want to see cooler images
(e.g. taken under bright daylights). Those effects can be automatically achieved by adjusting the color
temperature.
6.9.1 Color Temperature Browsing
6.9.1.1 Feature Extraction
The (correlated) color temperature of the scene-illumination in the image is extracted as follows.
Note: In this section, several references are made to sRGB, perceived illuminant, and (correlated) color
temperature and its reciprocal scale. All information on these subjects can be found in [AMD1-1][AMD1-2]
[AMD1-3][AMD1-4][AMD1-5].
6.9.1.1.1 The Overall View of Color Temperature Extraction Algorithm
1) Linearizing input image: RGB � R G B
l l l
2) Converting R G B into XYZ
l l l
3) Removing pixels that have the pixel value smaller than the low luminance threshold(T )
ll
4) Averaging XYZ value for all remained pixels: X Y Z
a a a
5) Calculating the self-luminous threshold: X , Y ,Z If X , Y ,Z have the same values with the
T T T T T T
s s s s s s
previous values, go to procedure 7), else remove pixels that have the pixel value bigger than the self-
luminous threshold and repeat procedure 4) to 6)
6) Averaging XYZ value for all pixels remained, estimating it as the illuminant tri-stimulus values, and
computing the scene-illuminant chromaticity coordinates (x , y ) in CIE 1931 diagram
s s
7) Converting the scene-illuminant chromaticity (x , y ) into color temperature T
s s c
(1) Calculating the chromaticity coordinates (u , v ) in CIE 1960 UCS diagram from (x , y )
s s s s
(2) Finding two adjacent isotemperature lines from (u , v ) and obtaining the distance from those lines
s s
(3) Computing the correlated color temperature using the distance ratio
6.9.1.1.2 The Detail of Extraction Algorithm
1) Linearizing input image: Obtain the linearized R G B from the inverse gamma correction of the input
l l l
RGB, which is the gamma-corrected for display devices
© ISO/IEC 2004 – All rights reserved 5

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
Note, it is assumed that an input image RGB is a gamma-corrected non-linear sR’G’B’ in the range of
0~255(8bit) in the following equations.
' ' '
if R (i, j),G ()i, j ,B ()i, j ≤ 0.03928× 255.0 ,
sRGB sRGB sRGB
'
 
R sRGB()i, j
R ()i, j = ÷12.92
 
sRGB
 
'
G ()i, j 
sRGB
G ()i, j = ÷12.92 ,
 
sRGB
 
'
 
B ()i, j
sRGB
()
B i, j = ÷12.92
 
sRGB
 
' ' '
else R sRGB(i, j),G sRGB()i, j ,B sRGB()i, j > 0.03928× 255.0 ,
2.4
'
 
 () 
R sRGB i, j
+ 0.055
 
 


R()i, j = R ()i, j =  
l sRGB
1.055
 
 
 
2.4
'
 
 
G ()i, j
sRGB
+ 0.055
 
 
 
 
G()i, j =G (i, j) = ,
l sRGB
1.055
 
 
 
2.4
'
 
 () 
B sRGB i, j
+ 0.055
 
 
 
B()i, j = B ()i, j =  
l sRGB
1.055
 
 
 
where (i,j) is the index for pixels
2) Converting linearized R G B into CIE 1931 tristimulus XYZ with conversion matrix M
l l l
X()i, j R()i, j
   
l
   
Y()i, j = Μ • G()i, j ,
l
   
Z()i, j  B()i, j 
l
   
0.4124 0.3576 0.1805
 
 
where conversion matrix M = 0.2126 0.7152 0.0722 .
 
0.0193 0.1192 0.9505
 
3) Removing pixels that have the pixel value smaller than the low luminance threshold(T )
ll
Y(i, j) 
ll
,

otherwise,   p(i, j) = 255

6 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
where p(i, j) is the label for each pixel at the location (i, j).
4) Averaging XYZ value for all pixels remained, which have p(i, j) = 255 : X Y Z . row * col intuitively
a a a
means the number of all pixels remained.
row−1col−1
X = X (i, j),
a ∑∑
(row×col)
i=0 j=0
row−1col−1
Y = Y(i, j),
∑∑
a
(row×col)
i=0 j=0
row−1col−1
Z = Z(i, j).
a ∑∑
(row×col)
i=0 j=0
Calculating the self-luminous threshold: X , Y ,Z
T T T
s s s
X = f ×k ×X ,
T a
s
Y = f ×k ×Y , ,
T a
s
Z = f ×k ×Z .
T a
s
where f*k* X Y Z means the estimated illuminant level [AMD1-4].
a a a
6) If X , Y ,Z have the same values with the previous values, go to procedure 7), else remove pixels
T T T
s s s
that have the pixel value bigger than the self-luminous threshold and repeat procedure 4) to 6)
If (X (t) = X (t −1), Y (t) =Y (t −1),Z (t) = Z (t −1) ) { go to 7) }
T T T T T T
s s s s s s
else {
X (i, j) > X or Y(i, j) >Y  or  Z(i, j) > Z ,   p(i, j) = 0

T T T
s s s

otherwise,                             p(i, j) = 255

}
repeat 4) ~ 6)
where t means the iteration time for the T and the initial values are set to
s
X (0) = 0, Y (0) = 0,Z (t) = 0 .
T T T
s s s
7) Averaging the XYZ value for all pixels remained, estimating it as an illuminant tri-stimulus value, and
computing the scene-illuminant chromaticity coordinates (x , y ) in CIE 1931 diagram. Again, row * col
s s
intuitively means the number of all pixels remained, which have p(i, j) = 255.
© ISO/IEC 2004 – All rights reserved 7

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
row−1col−1
X = X (i, j),
s ∑∑
(row×col)
i=0 j=0
row−1col−1
Y = Y (i, j),
s ∑∑
(row×col)
i=0 j=0
row−1col−1
Z = Z(i, j).
s ∑∑
(row×col)
i=0 j=0
X
s
x = ,
s
X +Y +Z
s s s
Y
s
y = .
s
X +Y +Z
s s s
8) Converting the scene-illuminant chromaticity (x , y ) into color temperature T .
s s c
(1) Calculating the chromaticity coordinates (u , v ) in CIE 1960 UCS diagram from (x , y ).
s s s s
4x
s
u = ,
s
− 2x +12y + 3
s s
6y
s
v = .
s
− 2x +12y + 3
s s
(2) Finding two adjacent isotemperature lines [Mori et al (1968)] from (u , v ) and obtaining the
s s
distance from those lines: if (u , v ) is located between i-th and i+1-th isotemperature line then di / di+1
s s
< 0
(v −v ) −t (u −u )
s i i s i
d = ,
i
(1+t )
i
where (u, v ), t: chromaticity coordinates and slope for representing the i-th isotemperature line
i i i
(Table AMD1-1 - Isotemperature lines: Calculated in accordance with the method proposed by Mori
et al.(1968): The color temperatures between 1667K and 25000K and corresponding parameters(u,
i
v, t ) are marked with blue fonts) and d : distance between (u , v ) and the ith isotemperature line.
i i i s s
(3) Calculating the correlated color temperature using the ratio of distance
−1
 
 
1 d 1 1
i
T = +  −  ,
c  
 
T d −d T T
i i i +1  i +1 i 
 
where T is the color temperature for the cross point of the i-th isotemperature line with the daylight
I
locus. The color temperatures less than 1667K and larger than 25000K are tuned to 1667K and
25000K, respectively.
8 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
Table AMD1-1 — Isotemperature lines: Calculated in accordance with the method proposed by Mori et
al.(1968)
i Reciprocal TemperatureT
u v t
i i i
Megakelvin (K)
1 0 Infinity 0.18006 0.26352 -0.24341
2 10 100,000 0.18066 0.26589 -0.25479
3 20 50,000 0.18133 0.26846 -0.26876
4 30 33,333 0.18208 0.27119 -0.28539
5 40 25,000 0.18293 0.27407 -0.30470
6 50 20,000 0.18388 0.27709 -0.32675
7 60 16,667 0.18494 0.28021 -0.35156
8 70 14,286 0.18611 0.28342 -0.37915
9 80 12,500 0.18740 0.28668 -0.40955
10 90 11,111 0.18880 0.28997 -0.44278
11 100 10,000 0.19032 0.29326 -0.47888
12 125 8,000 0.19462 0.30141 -0.58204
13 150 6,667 0.19962 0.30921 -0.70471
14 175 5,714 0.20525 0.31647 -0.84901
15 200 5,000 0.21142 0.32312 -1.0182
16 225 4,444 0.21807 0.32909 -1.2168
17 250 4,000 0.22511 0.33439 -1.4512
18 275 3,636 0.23247 0.33904 -1.7298
19 300 3,333 0.24010 0.34308 -2.0637
20 325 3,077 0.24702 0.34655 -2.4681
21 350 2,857 0.25591 0.34951 -2.9641
22 375 2,677 0.26400 0.35200 -3.5814
23 400 2,500 0.27218 0.35407 -4.3633
24 425 2,353 0.28039 0.35577 -5.3762
25 450 2,222 0.28863 0.35714 -6.7262
26 475 2,105 0.29685 0.35823 -8.5955
27 500 2,000 0.30505 0.35907 -11.324
28 525 1,905 0.31320 0.35968 -15.628
29 550 1,818 0.32129 0.36011 -23.325
30 575 1,739 0.32931 0.36038 -40.770
31 600 1,667 0.33724 0.36051 -116.45
© ISO/IEC 2004 – All rights reserved 9

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
6.9.1.1.3 Optimal Interval Determination for Color Temperature Browsing Categories
To find the optimal range of color temperature values for each browsing category, interval classifiers for fuzzy
categories based on rough information systems were used.
6.9.1.2 Browsing Method
1. For the hot image browsing, the browser starts displaying the images from the lowest sub-range in the
hot color temperature range and continues displaying the images in the subsequent sub-ranges.
2. For the warm and moderate image browsing, the browser starts displaying images in the middle sub-
range and continues displaying the images in the sub-ranges near the middle sub-ranges.
For the cool image browsing, the browser starts displaying the images from the highest sub-range in the cool
color temperature range and continues displaying the images in the subsequent sub-ranges in a descending
order.
The following is a pseudo-code in HTML format for color temperature browsing in the web browser using DOM and
JavaScript. This code reads the XML document and generates corresponding DOM objects. Assume that the XML
document of image DB is composed of image elements, which are again composed of an image link and its color
temperature browsing type. This code produces category buttons on the web window. If one of the 4 category buttons is
pushed, it will return the images belonging to the chosen category. Here, SortAscendingOrder(), SortDecendingOrder(),
and RearrangeNeartoFar() functions are left out to implementers where one can easily implement them.



color temperature browsing

Color Temperature Browsing

HOT   
WARM   
MODERATE   
COOL

---

© ISO/IEC 2004 – All rights reserved 11

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
6.9.2 Display Preference Control
People would often like to change the quality of the image or scene shown in display devices toward a way, which he/she
prefers most. For example, people often control several buttons such as hue, brightness, and contrast to get the most
natural and preferable scene for them. The color temperature provides the effective and efficient way to support the
display preference control.
6.9.2.1 Decoding of Color Temperature
Color temperature of image could be obtained as follows:
Decode the first 2bits to identify the color temperature range of the category (e.g. 00: hot -> [1667K, 2250K],
Tlb = 1667, Tub = 2250).
6 6 6 6
1) RTlb = 10 / Tlb, RTub = 10 / Tub (e.g. RTlb = 10 / 1667 = 599.88, RTub = 10 / 2250 = 444.444).
2) Uniformly quantize [599.88, 444.444] into 64 sub-ranges.
3) Decode the last 6bits (e.g. 000001 -> 2nd sub-range) and pick the corresponding range (e.g. [597.0149,
595.0227]).
4) Calculate a mean ((597.0149 + 595.0227)/2 = 596.0188) and a representative color temperature of the
range (i.e. 10 / 596.0188 ≈ 1678K).

6.9.2.1.1 Color Temperature Conversion for Display Preference Control
6.9.2.1.1.1 Color Temperature Conversion Overview
Input Image
User
Temperature
(RGB)
R ’G ’B ’ T
i i i u
R G B XYZ T
Linearize RGB → XYZ Calculate Temperature
i i i i
Color
(Inverse Conversion Temperature Mapping
Gamma) T
t
Calculation of the coefficient for Color
Temperature Conversion
M
c
Temperature XYZ → RGB
X’Y’Z’
XYZ
Conversion Conversion
R’G’B’
Gamma
Output Image R G B
o o o
correction
(RGB)
Figure AMD1-1 — Block diagram for the color temperature conversion
12 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
The overall flow of the color temperature conversion is shown in Figure AMD1-1 - Block diagram for the color
temperature conversion. The color temperature mapping functions, calculation of the coefficient matrix, and
image conversion are described in detail in the following subclauses.
6.9.2.1.2 Color Temperature Mapping Functions
When we obtain a user preferred color temperature against a training image (w/ a reference color
temperature) through some user interface, we need to calculate a target color temperature against the input
image on behalf of the relationship between the user preferred color temperature and the reference color
temperature. The color temperature mapping indicates the mapping from the input color temperature to the
output color temperature.
� Input
- T : Color temperature of input image
i
- T : User preferred color temperature
u
� Output
- T : Target color temperature
t
� Mapping Process
, user preferred color temperature T , and target color
1) Get the color temperature of input image T
I u
temperature T .
t
2) Determine the reference color temperature T , e.g. D65 (6500K).
r
3) First of all, we can make T become T Then, the color temperatures around T (i.e. bigger or smaller
r u. r
color temperatures around the reference color temperature) are converted in accordance with the
mapping function. An example is shown in figure below.
T T T
t_1 r t_2
T T
min max
T T T T T
min t_1 u t_2 max
When an observer wants to see cooler scenes, the images in moderate or cool category become cooler.
The images in warm or hot category would move a little bit towards cooler temperature or change none. The
opposite is also similar. When an observer wants to see warmer scenes, the images in moderate, warm,
and hot category become warmer. On the other hand, the images in cool category would move towards
warmer temperature or change none. We applied this scheme to the color temperature mapping method as
follows.
© ISO/IEC 2004 – All rights reserved 13

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
Let the training image’s color temperature be T, let T be the color temperature of the training image
r u
changed by the user’s preference, and RT and RT are the reciprocal color temperature of them,
r u
respectively, then let RT = RT – RT . Now the test image comes in with Color Temperature T and its
delta r u i
reciprocal RT , then we will set the target temperature, denoted by T , using the following rule.
i t
1) Calculate a weight Θ
When the color temperature chosen by a user is greater than the color temperature of the training image
(Tu > Tr),
Θ
If (T < T )  = 0,
i lctb
else if (T <= T <= T )
lctb I r
()T −T
i ctb
T = ,T =[]0,1
ni ni
()T −T
r ctb
Θ =T ,
ni
else
Θ
= 1.
T can be obtained as follows:
lctb
RT = RT + (RT –RT ) / 2 where RT = 10 /T
LCBT r min r
T = 10 / RT .
LCBT LCBT
When the color temperature chosen by a user is less than the color temperature of the training image (T <
u
T ),
r
Θ
If (T <=T )  = 1,
i r
else if (T < T <= T )
r I uctb
α
()T −T 
max uctb
Θ =   .
ucbt
 
()T −T
 max r 
With two known points, (T , Θ ) and (T , 1), a line equation y=mx+c can be obtained.
uctb uctb r
m=(Θ – 1)/( T – T ), c=1 – m* T ,
uctb uctb r r
Θ =m ∗T +c ,
i
else
()
T −T
ri i
T = ,T =[]0,1 ,
ni ni
()T −T
max rr
14 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
α
Θ = f (T ,α)(= T ) .
pow ni ni
T can be obtained as follows:
uctb
RT = RT – (RT –RT ) / 2 where RT = 10 /T
UCBT r r max
T = 10 / RT .
UCBT UCBT
2) Calculate a target temperature, T .
t
RT = RT – (RT x Θ),
t I delta
T = 10 / RT .
t t
6.9.2.1.3 Calculation of Coefficients of Color Temperature Conversion and Obtaining Converted Image
6.9.2.1.3.1 Color Temperature Conversion Coefficients
The color temperature conversion coefficients are members of the matrix that is used for converting between
two different color temperatures. In order to calculate the coefficients, we need a color temperature of the
input image (Ti) and a target color temperature (Tt) that is obtained from the mapping function. We can then
make the coefficient matrix M for the color temperature conversion. We have used the Bradford chromatic
c
adaptation transform for the color temperature conversion that is from CIE (commission Internationale de
l'Eclairage).
The process to calculate the coefficients is as follows.
Calculate XYZ for T and T . We can transform a color temperature T to a chromaticity (x, y) with the following
I t c
method. The chromaticity (x, y) can be transformed to tristimulus XYZ by giving 1 to the value of Y.
If (1667<=T < 4000K)
c
9 6 3
10 10 10
,
x = −0.2661239 − 0.2343580 + 0.8776956 + 0.179910
D
3 2
T T T
c c c
else if (4000K <= T <= 25000K)
c
9 6 3
10 10 10
,
x = −3.0258469 + 2.1070379 + 0.2226347 + 0.24039
D
3 2
T T T
c
c c
if (x <= 0.38405)
D
3 2
,
y = 3.0817580 x − 5.8733867 x + 3.75112997 x − 0.37001483
D D D D
else if (x <= 0.50338)
D
3 2
,
y = −0.9549476 x −1.37418593 x + 2.09137015 x − 0.16748867
D D D D
else
© ISO/IEC 2004 – All rights reserved 15

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
3 2
.
y = −1.1063814 x −1.34811020 x + 2.18555832 x − 0.20219683
D D D D
X =()x y
w D D
Y =()y y .
w D D
Z =()z y =()(1− x − y) y
w D D D D D
When T is transformed to (X Y Z ) and T is transformed to (X Y Z ), we can use the Bradford chromatic
I iw iw iw t tw tw tw
adaptation transformation to define the conversion between two tristimulus values.
−1
X X
       
tw iw
       
Y = M D M Y .
tw BFD BFD iw
       
Z      Z 
tw iw
       
Here,
−1
0.8951 0.2664 − 0.1614
   
   0.9870 − 0.1471 0.1600
   
   
, ,
M = − 0.7502 1.7135 0.0367
M = 0.4323 0.5184 0.0493
BFD
BFD
   
   
   0.0389 − 0.0685 1.0296    − 0.0085 0.0400 0.9685
       
R R 0 0
   
tw iw
   
D = 0 G G 0
tw iw
   
   
0 0 B B
   tw iw
In the matrix D, R , G , B and R , G , B are the cone responses of the each illuminant. We can calculate
iw iw iw tw tw tw
the con response as follows:
R X Y
    
w w w
    
G = M Y Y .
w BFD w w
    
B   Z Y 
 w    w w 
Finally, the coefficients of matrix M can be obtained as follows:
CT
−1
      
      
M = M D M
CT BFD BFD
      
      
      
16 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
6.9.2.1.3.2 Obtaining Converted Image from Input Image
Now, we can convert the input image to the output image with the target color temperature. First of all, we can
calculate the tristimulus XYZ of each pixel in the input image. Then, we can calculate the converted X`Y`Z` by
’ ’ ’ ’ ’ ’
multiplying the coefficient matrix M to XYZ. We can calculate R G B from X`Y`Z`. Finally, the R G B is
CT
transformed to R G B after applying the gamma correction process.
o o o
'
 () ( )
X i, j  X i, j 
 
'   
() () ,
Y i, j = M Y i, j
CT
 
  
'
 
Z()i, j  Z()i, j 
  
 
−1
' '
R()i, j  X (i, j)
 
l
   
'   '
G()i, j = M Y()i, j ,
l
   
 
' '
   
B()i, j   Z()i, j
l
 
   
' ' '
if (R l()i, j ,G l()i, j ,B l (i, j) ≤ 0.00304 ),
'
R()i, j = (R(i, j)×12.92)× 255
l
o
'
G()i, j =()G(i, j)×12.92 × 255 .
l
o
'
B()i, j =()B l(i, j)×12.92 × 255
o
' ' '
if (R()i, j ,G()i, j ,B (i, j) > 0.00304 ),
l l l
 
'
2.4
R()i, j = 1.055×()R − 0.055× 255
'l
o
 
 
 
'
2.4
G()i, j = 1.055×()G − 0.055× 255 .
'l
o
 
 
 
'
2.4
B()i, j = 1.055×()B − 0.055× 255
'l
o
 
 
6.9.3 DDL Instantiation Examples

hot
1

6.9.4 Conditions of Usage
This descriptor is useful when user wants a coarse browsing of images or video segments with similar
perceptual temperature feelings. Note that it can also be used in combination with the other descriptors in
MPEG-7. For example, in looking for a preferred image, one can use the color temperature descriptor to find a
set of candidates with the similar perceptual property or feeling without any query image and then use the
© ISO/IEC 2004 – All rights reserved 17

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
dominant color descriptor or the color layout descriptor to get a precise similarity match list among the
candidate images. The case is similar for video segments. In this case, a representative color temperature
descriptor value can be obtained by calculating the average color temperature descriptor values of all the
frames or by selecting a color temperature descriptor of a representative frame in the video segment.
This descriptor is also useful when user wants to change the display quality of still images, video or TV signal
in favor of the user’s preference. User can automatically change the color temperature of video or TV signal.
User can see a natural effect of changing illumination shone on the scene as long as maintaining the relative
color temperature difference between consecutive scenes. In real-life application, we assume that there is an
agent, in the form of software or hardware, to fulfil the functionality of display preference. The agent obtains
user’s preference for display through several possible interfaces and records it. The preferred color
temperature value can be obtained through a direct input from a user or through a user’s choice on given
several examples of visual images.

6.10 Illumination Invariant Color
This descriptor provides means to achieve illumination invariance with existing color descriptors; Dominant
Color, Scalable Color, Color Layout and Color Structure. This involves the preprocessing of the images to
normalize their color temperatures to canonical illumination 6500K. Its main functionality is the similarity
matching independent of illumination changes including the presence of shadow.
6.10.1 Feature Extraction
To achieve illumination invariance, the descriptors for both the query and database images are extracted from
preprocessed images. The preprocessing consists of the Bradford chromatic adaptation transform defined in
detail in the subclause 6.9.2.1.3.1. The transform converts the averaged chromaticity diagram of an image to
the chromaticity diagram of canonical illumination corresponding to 6500K on the daylight locus. Once this
preprocessing is applied, the color descriptors are extracted and matched as specified in the corresponding
subclauses
Tool Subclause
Dominant Color 4.2.3.1
Scalable Color 4.2.4.1
Color Layout 4.2.5.1
Color Structure 4.2.6.1
The algorithm for estimation of illumination (x, y) of an image is a combination of the color temperature
extraction algorithm in the subclause 6.9.1.1.2 and the Grey-World algorithm. Its detailed procedure is as
follows.
1) The preprocessing step.
This is the same as the first 2 steps of the color temperature extraction algorithm as described in the
subclause 6.9.1.1.2. They consist of the following 3 steps.
� Convert image pixel RGB values to XYZ values.
� Remove the low luminance part.
� Remove the self luminance part by iterative manner.
2) The illumination (x, y) estimation step.
� Project the remaining XYZ values to x-y values.
� The bin counting step.
18 © ISO/IEC 2004 – All rights reserved

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
- Divide the x-y plane in 60 x 60 grids. We’ll call the each cell as the ‘bin’.
- Index each bin 0 or 1 according to the presence of the (x, y) values in it, with a proper
noise removal scheme. Specifically, there’ll be a threshold value alpha depending on
the image size and the total bin number that, for each bin, if the number of (x, y)
values belonging to that bin is less than alpha, then the bin index is 0, otherwise the
bin index is 1.
� Average the center (x, y) values of each bin with index 1 to estimate the illumination (x, y).
This is based on the Grey-World algorithm.
3) Image conversion step.
It is done in the same way as explained in the subclause 6.9.2.1.3. It consists of the following two steps.
� The image XYZ pixel value conversion by the Bradford transform calculated between the
illumination (x, y) and the illumination (x, y) of the 6500K on the daylight locus lying in the x–y
plane.
� Convert XYZ values to RGB values.
After converting the image to the canonical illumination 6500K, one can extract color descriptors (i.e. dominant
color, scalable color, color layout, and color structure) from the converted image.
6.10.2 Similarity Matching
The similarity matching measurements from color descriptors for non illumination compensated pictures are
equivalently applicable to the illumination compensated pictures.
6.10.3 DDL Instantiation Examples


50
34
30
16 12 15 12 17
12 17
12 14


6.10.4 Conditions of usage
This descriptor is useful when user wants to retrieve images with similar contents taken under different
illumination conditions. In real-world, the illumination condition often changes due to varying weather
conditions, different image acquisition times, presence of shadows, etc. The properties of the artificial
illumination strongly depend on the lighting appliances used (e.g. bulbs, fluorescent lights). The color
descriptors defined in ISO/IEC 15938-3 are mostly for the similarity search and retrieval functionality with no
consideration of the illumination change in visual content. Illumination change can cause a great deal of
change in the color distribution of the image and often makes two images with similar scene lie far apart in the
color space. This descriptor acts as a container of the color descriptors so that any color descriptors can be
extracted after applying the illumination invariant process and used for the illumination independent similarity
matching. User will effectively retrieve images with this descriptor when the image database contains many
similar scenes taken under different illumination conditions.

© ISO/IEC 2004 – All rights reserved 19

ISO/IEC TR 15938-8:2002/Amd.1:2004(E)
Add after subclause 8.4:
8.5 Shape Variation
This descriptor describes the variation of shape in a collection of binary images of objects. The collection of
binary image includes a set of segmented images in sequence from a video. Its main functionality is to retrieve
similar collection of shape images regardless of their sequence or the number of frames in each collection.
When they are in sequences, it can be used for retrieval of Shape Variation represented by a set of frames in
video segments – in terms of the similar shape variation due
...