A Syndication Project

Syndication -- An Overview

This URL: http://www.iangraham.org/projects/news/overview.html

Created: 18 September, 2000
Last Update: 22 September, 2000

Author(s): Ian Graham, Benet Devereux

1. Overview

More and more Web applications depend on data distribution -- that is, the publication of data by a data provider and the gathering and processing of such data by a data consumer. There are many examples where this is done, including:

These different cases share a common feature -- one organization publishes 'origin' data and makes it available in some form, and another organization downloads the data and processes the data to integrate it in some way into their own database or application. However, the present systems for doing this use different and incompatible tools and protocols to do accomplish the same tasks. Thus, it seems sensible to try and develop a language that reflects the semantics of syndication independent of the details of the actual data being syndicated. Then, all syndication-related applications could be built using a single syndication language to encode the data for transport. Other protocols could be used to negotiate how data would be transmitted received (e.g., XML, RPC), while interally, a syndication management tool might use data content-specific processors (bound to the MIME or other data types of given components) to suitably process the content of the message.

Aggregating Diverse Data

What is important here is that it should be possible to manage these blocks of data should take place independent of detailed knowledge of the content of each block. That is, the recipient of a syndicated data message should be able to generally figure out what type of data each block of the message contains, and how to "manage" it as a chunk of syndicated data, without needing to "go inside" the data and extract information from it. This lets a syndicated data management system handle the task of managing the data without needing to proces the blocks themselves. This allows for for a highly flexible syndication system, since new data types can be added without affecting the syndication language (or syndication management software), while the syndication language can itself evolve without detailed reference to the data the message encodes.

Thus, this model is not tied to existing 'data' format such as XML-news, RSS and the like -- but the intent is that it should be easy, using XML namespaces, to include RSS, XML-News (or other) data as content of a syndicated data message.

The Main Issues

This type of separation means that a syndication message must itself contain enough mangement data that the syndication management process can succeed. Some of the main issues are:

2. References

  1. http://my.netscape.com/publish/help/mnn20/quickstart.html -- Netscape's Specification for Resource/Rich Site Summary, Versions 0.91/0.92
  2. http://purl.org/rss/1.0/ -- Proposal for RSS 1.0
  3. http://www.magnicomp.com/rdist/rdist.shtml -- Rdist is a tool for accessing and automatically retrieving files and other resources over the Internet. It is often used to automatically synchronize data content across servers.
  4. http://gnutelladev.wego.com/ -- Gnutella is a distributed information/data sharing technology.

Ian S. Graham