Next-generation product capabilities for DVD recorders, set-top boxes and digital camcorders will center on creating, sharing and consuming digital content, as the battle for leadership in the digital home intensifies. Jetstream Media Technologies' new JetFx™ IP core adds PC like video editing functionality such as real-time transitions, color effects, title and clip-art animation as well as the automatic creation of personal movies based simply on a user selected theme. Figure 1 is the block diagram of a sample DVD recorder application in which JetFx IP core is incorporated in the design to provide real-time video effects.
View full size
CE Design Issues
Within the CE market, several factors make it difficult to achieve success for new functionality. Features must be powerful but easy to use. Designs must be sophisticated but easy to integrate and cost sensitive. Time-to-market is critical so design reuse, partnerships, and flexibility are often the difference between a successful product and a missed opportunity.
From a designer’s perspective, an ideal video effect-generating device for the CE market has several desirable attributes:
- Support wide selections of video effects all in real-time
- Programmable to allow new effects to be easily introduced
- Support HD (1080i) resolutions for the next generation of products
- Industry standardized interfaces for quick integration with existing designs
- Efficient, low cost and standardized memory architecture
- Optimized for small silicon area and low power for cost sensitive applications
There are different approaches that designers could take to implement advanced video effect functionality on their CE devices. The two most common approaches would be to use embedded DSP or RISC processors.
These approaches however, have several important drawbacks that make them impractical for CE applications. First, they are often too costly given that amount of processing power required to perform even simple video effects. Secondly, they force the designer to write, debug and maintain complex software for video effect processing that may not be in their core competency.
A more viable approach would be to use application specific IP cores that can reduce software complexity, meet cost-performance targets, and offer the flexibility necessary for quick time-to-market.
Introducing the JetFx IP Core
The patent-pending JetFx IP core is designed to meet all these desirable attributes. With its ability to compute higher order functions, the JetFx IP core is capable of calculating many different mathematical functions while maintaining a small silicon size. The high order calculation uses only adders, not multipliers – fundamentally simplifying the complexity of the texture mapping.
The screen can be divided into several regions according to the special effect being applied. The JetFx IP core then calculates texture mapping for each region. The texture mapping is not only for simple 2D effects such as rotation and scaling, but also for non-linear texture mapping to approximate 3D perspective projection and warping. In addition to texture mapping, these mathematical functions are also used to define primitive elements that form complex shapes or gradient colored objects.
The JetFx IP core is fully parameterized to allow different performance and functionality options to be selected based on the target application. For price sensitive consumer products where cost can be a driving factor, a modest performance, less feature rich design can be selected. For high end consumer or professional products where performance is paramount, a full set of functionality and performance enhancing capabilities can be included such as anti-aliasing, high definition support, and title animation.
For complex visual effects, more sophisticated mathematic functions are needed. JetFx IP core assembles families of higher order complex mathematic functions by performing linear combination operations on a base set of simpler functions. By carefully selecting portions of few family member functions, many smooth functions can be formed on the fly for visual effect generation. In addition, the multiplications used by these linear combination operations can usually be simplified to shift and add operations by choosing appropriate base calculations. This helps reduce silicon complexity and size while creating new effects.
The Wine Bottle Effect
Next: Page flips, warp and cube effects
To improve the perspective-like projection, two multiplications may be included in the IP core to determine the ratios of functions for the pairs of opposite edges in video objects. Figure 2 illustrates four video effects created by JetFx™ IP core. They are cube effect, page flip, warp effect and geometry shapes definition.
View full size
When creating the JetFx™ IP core, several architecture and design decisions were made to increase the efficiency of the JetFx™ IP core. They are listed as follows.
- 4x pixel rate – Due to the many mathematical functions involved in creating video effects, the JetFx IP core throughput needed to be several times faster than the actual pixel rate to provide sufficient operational headroom. This is typically done by either operating the standard JetFx IP core at 4x the pixel rate or alternatively using 2 sets of JetFx™ logic at 2x the pixel rate.
- Shared state machine design – Treat each incremental evaluation of a mathematical function as a state machine with state variables. State variables for each function are stored in memory called a State RAM.
- Multiple region support – Regions are screen areas separated by lines. JetFx IP core detects regions automatically and switches to different sets of mathematical functions for each region.
- Bounding box support – Mathematical functions are evaluated in their own bounding box. This is helpful for sharing the computational resources and to prevent overflow or underflow during computation of higher order functions.
- Stackable effect support – Multiple effects are stackable. To ensure real time video effect creation, different effects can be created using different computational resources inside the JetFx IP core. For very sophisticated video effects, JetFx has the option of generating the effect in a multiple-pass acceleration mode.
JetFx receives video effect definition data on a per video field basis on the host side. On the frame buffer side, JetFx access the video source frame, clip art, and overlay graphics, title from the external SDRAM through a SDRAM controller shared with other system modules such as MPEG video and audio encoders, NTSC/PAL Decoder, DV codec, etc.
One of the biggest challenges was how to allow the IP to be integrated across a wide variety of different products each with their own design goals and architectures. The JetFx interface challenges were:
- Support for both SD and HD (1080i) meant bus bandwidth and memory access bandwidth can change by a factor of six
- Need to interface to several types of on-chip bus interface protocols
- Support for many different target applications (e.g. DVD-Recorder, Camcorder, Video Playback Devices, Capture Devices, etc.) and as a result, the need to interface with widely different system components
The Broken Glass Effect
Next: OCP for interface flexibility
OCP Interface Protocol
To maximize the interface flexibility, OCP standard interface protocol was adapted by the design team. Figure 3 illustrates a sample application of JetFx™ in video systems. It uses two OCP interfaces, a master and a slave. On the host side, JetFx is an OCP slave. In this illustration, the host sends effect definition data to JetFx IP core via DMA controller. On the frame buffer side, JetFx is an OCP master. It requests read and write accesses from / to frame buffers stored inside SDRAM through a memory controller.
View full size
For a SoC design team, the available bridge between OCP and many on-chip system buses make OCP-compliant IP cores more attractive as it is adaptable not only to their current SoC project, but also to those SoC projects in the future where system requirements and market conditions are continually evolving. Since all OCP signals are unidirectional and synchronous, this allows the design team to tailor each interface implementation, such as the data bus width or bus handshake, without modifying the IP core.
From SoC project management standpoint, no time penalty will be incurred to use the IP in multiple projects across different product lines. This “plug and play” approach of OCP design, not only lowers risk and cost, but also helps to shorten critical time-to-market.
From IP core developers’ point of view, the core is more likely to be modified for various SoC projects due to the diverse applications possible with the JetFx IP. This results in multiple versions of the same IP core, with multiple test benches, and test vector sets.
This situation can be eliminated, however, by using OCP on-chip interconnect technology. Because the IP core is not tied to any CPU architectures or their buses technology, the core remains untouched and only one IP core, one test bench and one test vector set needs to be maintained. In addition, since all OCP signals are unidirectional, synchronous, and configurable, during implementation of the IP core, they can be easily customized to match core service bandwidth and latency requirements.
In conclucsion, the JetFx IP core is ideal for companies who need to keep costs down, time to market short, while adding more features and functionality to the next generation of consumer devices for the digital living room. Its programmable architecture gives a design more headroom so that numerous new video effects can be implemented in future CE devices.
By incorporating an OCP interface the designer is assured of decreased integration issues and the ability to easily move the IP core to different designs without requiring extensive new test and verification procedures. The multitude of different buses, CPU architectures, and interface parameters can make design reuse and its benefits of reduce costs as well as improved time-to-market, difficult to balance. OCP has helped resolve these issues for the JetFx™ IP core.
Animations shown here are sample transition effects. You can view large-size animated examples of many more video transition effects, and sample edited home movies at the JetFx Gallery: http://www.jetsmt.com/us2/product1-sub4.asp
The Falling Tile Effect
About the authors
Chuck Schalm is the Director of Sales and Marketing for Jetstream Media Technologies. He holds a BSEE from Michigan Technological University and an MBA from Keller Graduate School of Management. He can be reached at .
Dr. Philip Chao is the President and Founder of Jetstream Media Technologies. He holds a PhD in Electrical Engineering from Northwestern University. He can be reached at .