In addition to the mini film score reviews, you will find some of my technical reviews of products for film composers and sound designers. In the pipeline is a series on using Apple Logic for film music and sound design and interviews with some of today's top film composers. Stay tuned and thanks for reading!
Sunday, March 20, 2016
Fingerprinting, Watermarking, Tune Sat and Competitrack
Fingerprinting is a term for software that performs fuzzy matching of audio or video content. Each technology implementation has its own “special sauce” for representing audiovisual content and for matching fingerprints of unknown content against a database of previously extracted fingerprints. The fingerprint can be thought of as a summarization, or signature of the recorded audio or video that is robust against natural degradations such as lossy compression and frequency or temp changes.
Attributes that go into an audio fingerprint can include tempo, dynamics, general frequency patterns, segment structure (verse/chorus/bridge) and specific patterns in rhythm, melody and harmony (chords) in relation to the timeline of the audio.
Shazam is a company that built a business around a technology that is optimized to identify sound recordings in noisy environments. Their flagship application provides instant track recognition on mobile devices. Another company, Audible Magic uses their fingerprint identification for anti-piracy purposes. There are many other technologies; some are specialized in identifying brief snippets of a recording — suitable for airplay monitoring. Other technologies require a complete song recording to make the identification, and are therefore more suited to digital asset identification than to performance monitoring.
A fingerprint identification service begins with a set of audio files and metadata describing the content. Each audio file is fingerprinted and the fingerprint is stored in a database along with metadata, which could be artist, title, ISRC, and copyright holder information in the case of music tracks. If the tracks being indexed are commercials, the metadata might include the company name, product name and the Industry Standard Coding Identification (ISCI) for each ad.
Fingerprinting technology is frequently used for airplay monitoring. Automated listening stations can be configured in the desired geographic areas and computers can be configured to monitor multiple stations simultaneously. The monitoring software applies the fingerprinting algorithm to the broadcast signal and compares each short segment of airplay against the database of known fingerprints. When a match is found, a record is stored in a database indicating the date/time, channel and the identifier for the audio that was matched.
Whereas fingerprinting technologies do not change an audio file in any way, watermarking technologies do change an audio track (or video) by adding a perceptually inaudible (or invisible) signal. The signal carries the watermark payload — a set of digital bits. A watermark detector is a piece of software that “listens” to an audio recording or stream, say a broadcast, and extracts the payload from a detected signal.
Watermarks payloads are measured in terms of the number of bits that can be represented. In practice, a music label or film studio can watermark their content forensically using a different payload depending on the supply chain. For example, a hypothetical payload of “12345” might be used for all audio sent to a particular CD pressing plant and a payload of “67890” is used for audio sent to a digital fulfillment partner. If a piracy leak were found before the product is released, the watermark would identify which path in the supply chain resulted in the leak.
Forensic uses of watermarks also include “serial” watermarks, where each copy of a file or audiovisual stream is watermarked individually for each consumer or each performance. For example, the watermarks used in Digital Cinema not only identify the theater but also the date and time of the screening. When a “CAM” of a movie is found on the Internet — for example, on YouTube — the watermark identifies the source of the recording.
TuneSat is an independent audio monitoring services. They claim that their fingerprinting technology is used to monitor hundreds of TV channels. Their customers include organizations and individuals who desire an independent survey of broadcast performances of their audio content, including recorded music and commercials.
TuneSat claims to have listening stations worldwide. Each monitored performance includes channel, country, show name, date/time, descriptive metadata and a link to a downloadable recording of the broadcast. One limitation of their service is that it is focused on national broadcasts. Smaller, local broadcast markets are not monitored.
The benefit for rightsholders is that data from TuneSat can be used to dispute royalty statements from PROs that may not account for all performances. Another application for the TuneSat service is monitoring for unauthorized uses. An example would be commercials or shows that use a sound recording without a master use license.
Competitrack is a New York City based company providing ad tracking and business intelligence services across 22 media channels, including TV, print, radio and online ads. They were acquired by Market Track, LLC in 2013.
Competitrack claims to have data on 11 million ads that have run in the US and over 60 countries. They offer their customers information on ad spending by their competitors. Customers of Competitrack can view ad images across various media to learn where their competitors are running ads and how much is being spent on ads in different markets and across different media outlets.
In 2007, Competitrack announced that they had acquired an “advanced video fingerprinting technology,” which can be used to monitor video piracy. They claim their technology combines audio and video luminescence pattern recognition that can find a match with as little as 5 seconds of content.
Competitrack says their fingerprint technology has been used by most of the PROs in the US. In 2007, SESAC announced their in-house tracking system, which leverages data from Competitrack’s fingerprint technology along and DigiSound, a watermark-based tracking system for production music. SESAC points out that fingerprint and watermark technologies improve the accuracy of performance tracking without requiring their members to provide “proof-of-performance via media buys, traffic schedules or station logs.”