[Tig] How to search for video artifacts?

Martin Holmes martinh
Sat Nov 24 14:57:50 GMT 2001

Before you can make any kind of subjective assessment using an audience
of non-calibrated eyeballs you need to create an acceptable form of
objective measure to use as a yardstick for the subjective measurements.

There has been massive amounts of very creative work done to establish
these objective picture quality measures for MPEG2 coded systems.
Indeed, under the auspices of the ITU's VQEG (Visual Quality Experts
Group), a determination was made that PSNR (Peak Signal to Noise Ratio)
was still the most significant method of ascertaining benchmark picture
quality.  Any objective quality measure should correlate with the
measure of PSNR.

I can supply you with an interesting paper on the subject by the
engineers at S&W.  I won't subject everybody on the list to it, just
send me an e-mail and I will send it on to you.

Of course, this only works for MPEG2 coded data.  Use of bespoke or
non-standardized data coding systems would require complete testing from

Thanks, MH

-----Original Message-----
From: Chuck Harrison [mailto:chuck_harrison at iname.com]
Sent: Thursday, November 22, 2001 6:58 PM
To: tig at tig.alegria.com
Subject: [Tig] How to search for video artifacts?

Ladies & Gents:

A major international organization is planning to do some critical
tests on advanced, very-high-quality, compression methods. These
tests will undoubtedly comprise several different types of
experiments; I am discussing just one type here.

The goal of these experiments is to characterize the particular
artifacts that arise in different types of stressful video material.
Video that has been processed through a compression coder-decoder
("codec") will be compared directly against the pristine original.
All data will be in digital form so the uncertainties of analog
transmission and recording are eliminated, and the true
characteristics of the codec algorithms will be made visible.

The crux of the experimental design comes down to "What methods
are best -- most sensitive -- for revealing subtle differences
between the original ("A") and the processed ("B") video?" Many
experts have already contributed ideas and here are some of them,
some of which might be used in combination:
(1) Sequential: show the A clip followed by the B clip
(2) Side-by-side: A left, B right, two screens
(3) Splitscreen side-by-side: Left half of A splitscreen with
     Left half of B
(4) Splitscreen butterfly: Left half of A splitscreen with
     Left half of B *mirrored* (mirror matchline at center screen)
(5) Video difference: subtract the A video data from the B
     video data and display that
(6) "Trick play": slow motion forward and back; freeze frame
(7) Gain and pedestal: Post-process both A and B with identical
     gain or offset boost to reveal certain tonal ranges
(8) Sliding splitscreen: Wipe between A & B during running
     footage or on freeze frame

As a result of this type of testing, a qualitative evaluation
should be possible (e.g. "We see subtle mosquito noise during
a slow pan on this high-contrast object.") This type of
information is believed to be very valuable in improving the
design of these advanced compression codec algorithms.

I am sure that several readers of this list have already
contributed to the test-design effort. However, it may be that
additional useful suggestions on test methods (and on
concrete ways to execute them) could come from the
remarkable group on this list.

So tell us, what would *you* do to look at compression
artifacts under a microscope, as it were?

  Chuck Harrison
  Far Field Associates, LLC
  +1 360 863 8340 (voice)  PST = GMT-0800

Thank you Joe Moore of daVinci for supporting the TIG.

No marketing or advertising on the main TIG, 
send submissions to tig-announce at tig.alegria.com
Tig mailing list
Tig at tig.alegria.com

More information about the Tig mailing list