Most Read This Week
Web Conferencing Using the Java Media Framework (JMF)
Broadcast and receive media streams
Aug. 5, 2004 12:00 AM
This article describes our experiences with developing a browser-based Web conferencing application with the following constraints:
The article will start with a brief explanation about real-time streaming, followed by an introduction to JMF, a description of the capture and player applets, and a comparison with other technologies.
Introduction to Real-Time and Progressive StreamingReal-time streaming of media allows users to play media as they receive it. Users don't have to wait for the whole media file to be downloaded before watching. To enable real-time streaming, dedicated streaming media servers and streaming protocols, such as RealTime Protocol (RTP), are required. RTP is an Internet standard for transporting real-time data. It uses the unreliable UDP protocol to transmit packets.
A variant of real-time streaming is progressive streaming, also called HTTP streaming because it uses the commonly used HTTP protocol and standard HTTP servers to deliver media files. Progressive streaming enables files to be watched as they are downloaded. When the client makes a request (HTTP) to the server for the media file, the file eventually gets stored in the client's memory buffer. The playback is allowed before the entire file gets downloaded. Most firewalls allow traffic over HTTP whereas RTP is not permitted by most firewalls. In our approach, we're emulating HTTP streaming.
Introduction to Java Media FrameworkThe JMF API specifies a simple, unified architecture to synchronize and control audio, video, and other time-based data within Java applications and applets. JMF software, documentation, sample programs, and the source code can be downloaded from Sun's Web site at java.sun.com/products/java-media/jmf. In this section, we'll briefly discuss the basic concepts of JMF, including a few useful classes required to build a Web conferencing application:
1. The DataSource class is an abstraction that represents audio, video, or a combination of both. A data source can be a file or a stream and is constructed from the Manager and MediaLocator as follows:
DataSource ds = javax.media.Manager.createDataSource(mediaLocator);
Here, MediaLocator is a class that JMF uses to represent audio or video media location and is created as follows:
MediaLocator mediaLocator = new MediaLocator("vfw://0");
2. The Player class is used to play media files or stream media. A player is constructed from MediaLocator or the media URL as follows:
Player player = Manager.createPlayer(mediaLocator);
Once the player is realized (ready to play state), you can call player.start() to play the media. A realized player can be created from the DataSource:
Player player = Manager.createRealizedPlayer(ds);
3. A processor is a type of player. Besides playing the media, it can also output media through a DataSource to another player or processor. A processor is used to manipulate the data and convert the data from one format to another. It's created from the DataSource, MediaLocator, or a URL:
Processor processor = Manager.createProcessor(new URL("http://localhost/test.mov));
4. A manager is one of the most important classes of JMF. It handles the construction of players, processors, and DataSources, as we have seen earlier.
Architecture DescriptionThe architecture of our approach is shown in Figure 1. It implements a Web conferencing application over HTTP. The architecture has one centralized server and one or many distributed clients. The server has a Web server and a servlet container. Clients run two applets, one for capturing media and the other for playing the media.
The high level steps are:
1. The applet continuously captures video and audio streams from the Webcam. These streams are saved locally in a specified format as a file every few seconds. This file is uploaded to the server over HTTP using a file upload servlet. Uploading uses a separate thread. A significant loss of frames will result if the file upload is in the same thread as file capture. Note that a more efficient method would be to write these streams directly on the server using a socket. This is currently not possible because the DataSource class provided with JMF does not contain a method to get the InputStream. A custom InputStream-based DataSource can be developed (e.g., www.extollit.com/isdsjmf.php).
2. A server gets a new file clip from the sender and stores it in a sender-specific directory. A counter, such as filename+i, is attached to the filename.
3. The JMF Player applet continuously downloads new files from the Web server. It uses JMF's perfecting capability to play these clips in a continuous manner. When the current clip is being played, a new instance of Player is created for the next clip and the next clip is downloaded from the server. This makes the playing of clips continuous, as the next clip to be played has already been prefetched. Note that the entire clip is downloaded by the player applet before playing it.
At the start of playing and during the process of fetching a new clip, the player applet checks new file availability for n seconds before timing out.
Computation of ParametersFirst, we'll do an approximate mathematical analysis for bandwidth consideration and demonstrate the usability of our approach with a few special cases.
One second file clip size = oneSecFileSize bits
If the time to upload or download a clip is more than the time to play a clip, the player will wait and the receiver will see a break, i.e., max(tUpload,tDownload)>cSec. For the continuous playing of clips, the following condition must be true:
Max (1/uRate, 1/dRate) > 1/ oneSecFileSize
According to the equation, the wait time between clips at the receiver does not depend on clip size. The only variable that matters for a continuous playback is the size of a one-second file and that the provided upload and download rates meet the above condition. Lag time between playing and capturing is:
cSec + tupload + tdownload
From the above equation, the maximum lag with no break in the feed is 3*cSec, and the minimum lag is cSec.
To get a Web conference that is as close to real time as possible, cSec should be reduced. Next, we will apply the above analysis to the following cases.
Let's assume the uRate = dRate = 20K bits/sec. In this case, the one-second file size should be less than 20Kbits. If the clip size is 10 seconds, the maximum playback lag will be 30 seconds. We have observed that the minimum file size for transmitting a one-second video (with no audio) is 8Kbits using H263 encoding and 128x96 pixels video size. H263 encoding is ideal for a low-bandwidth environment because it produces smaller file sizes. The H263 encoder in the JMF 2.0 is capable of handling only limited video sizes (only 352x288, 176x144, and 128x96). We observed a minimum file size with the video and an 8-bit mono audio with an 8000Hz sampling rate to be 80Kbits.
Let's assume that the lower rate is rate = 20Kbits/sec and the other rate is much higher. In this case the one-second file size should be less than 20Kbits, but the maximum playback lag is about 20 seconds if the clip size is 10 seconds.
In this case better quality video can be transmitted. The playback lag will be the same as the clip size in seconds. JPEG encoding offers good quality video and is well suited to a high-bandwidth environment. File sizes can be decreased during JPEG encoding by lowering JPEG quality.
There are no easy guidelines to predict the exact size of the one-second clip; it depends on the video size, the audio sampling rate, video and audio encoding, the frame rate, and the file format. Users should experiment with using different values for these parameters and a variety of movement in the video to determine an approximate one-second file size.
JMF Capture Applet
JMF Player Applet
HTML Code for Sender and Receiver Applets
The HTML code for the receiver applet is shown in Listing 5. Note that this HTML page is generated dynamically with the appropriate senderID and current counter. If the receiver wants to receive multiple feeds, multiple applet entries are generated in HTML.
Thus far, we have described an HTTP-based approach that involves no real-time streaming. An alterna-tive to the above approach is a peer-to-peer, RTP-based Web conferencing solution that can be developed using the JMF API. The source code for the RTP Server/Sender can be found at java.sun.com/products/java-media/jmf/2.1.1/solutions/AVTransmit.html and at java.sun.com/products/java-media/jmf/2.1.1/samples/sample-code.html#RTPPlayerApplet for the RTP Player applet. The RTP Server captures the media from the Webcam and streams it to receivers by specifying IP addresses and port numbers. The RTP Player listens on a specific port for streams coming form the sender's IP address.
The primary difference between the HTTP approach and the RTP approach is that RTP streams the feeds continuously to receivers without storing them in files locally or at the server. The disadvantages of the above RTP approach are:
The architecture of the Web conferencing system using commercially streaming servers is shown in Figure 2. Senders first register a unique broadcast/mount point with a user management component as shown by arrow 1. The sender then uses streaming protocol (for example, RTP or RTSP) to push the media stream to a centralized streaming server, as shown by arrow 3. Receivers first look for senders at the user management component (as shown by arrow 2) and obtain corresponding broadcast addresses. Receivers then request and receive media from the streaming server, as shown by arrow 4.
In this approach we don't need to break the feed into smaller clips. Senders use the encoder to stream the media to the server. The server then streams the media to receivers. The disadvantages of this approach are:
ConclusionThe approach presented here offers a near real-time, low-cost Web conferencing solution. It allows multiple users to broadcast and receive media streams, it uses the HTTP protocol, and does not require the broadcaster and receiver to have public IP addresses. The source code for this article is available at www.indent.org/jdj-jmf/.
Reader Feedback: Page 1 of 1
Subscribe to the World's Most Powerful Newsletters
Today's Top Reads