Napster-like technology takes Web search to new level

First it was the record companies' nightmare. Now Yahoo and AltaVista might be next.

The loose group of open-source programmers responsible for the controversial Gnutella file-swapping software have turned their technology into what they say is a powerful new Web search tool.

By adapting the same technology now used to swap MP3 music files between hundreds of thousands of computer users, they say they've created a search engine that can be more up-to-date and potentially more targeted than search sites such as Yahoo or Lycos.

They've posted a rudimentary trial version--what they call a "proof of technology"--at InfraSearch. The bare-bones site is short on design and functionality at this point but provides a look at a search tool that goes beyond the capabilities of most existing Web search engines.

"This is just something we're doing to show what is possible," said Gene Kan, one of the programmers who has taken a lead on the project. "We want to prove to the world that Gnutella is more than just music piracy and child porn."

The group has been working largely in secret for the past few months, showing the project to just a few outsiders. One of those was Marc Andreessen, founder of browser software maker Netscape Communications, who now serves as chairman of Web infrastructure provider Loudcloud.

"It's a big deal," said Andreessen, who met with Gnutella developers last week and quickly became an admirer. "It will be a way for businesses to expose what they want people to find more easily."

It also is one of the first moves by what has been hugely controversial file-swapping software into the realm of unquestionably legitimate Web business. That's likely to take some of the legal shadows off the technology and could spur a new phase in development.

Open-source roots
Gnutella is one of several free software programs propagating quickly around the Net that are throwing chills into the collective spines of record executives, movie moguls and other copyright holders.

Gnutella originally was developed by programmers at Nullsoft, the online music company bought last year by America Online. But the project was shut down when it bleeped onto the radar of AOL, which is merging with music and film giant Time Warner.

The action came too late, however. A beta version of the program had already been distributed online, and several intrepid programmers snatched it up, reverse-engineered it, and distributed the blueprint information--or "source code"--to the Net.

Since that time, a community of unaffiliated developers, including Kan, have sprung up around the software, creating dozens of versions of the program for different computer platforms, all without AOL's or Nullsoft's participation.

Like the controversial Napster software, Gnutella allows hundreds or even thousands of people at a time to hook their computers together to share music libraries. Unlike Napster, however, it allows people to search for any kind of files; a random sampling of the search terms being used at any given time ranges from MP3s to blockbuster movies to pornography.

Also unlike Napster, it has no central point that serves as a directory. Each individual computer user connects to a few others, each of which in turn connect to a few more, and so on, creating a vast daisy-chain of connected computers.

This decentralized architecture means there is no company against which to file the kind of copyright-infringement lawsuit now facing Napster, a prospect that has worried record executives.

Kan and a handful of associates have created a kind of Gnutella portal to help guide the disparate development efforts and keep the various versions of the software compatible.

Building a better search engine
But Kan's team also had its eyes on something bigger, believing that the decentralized model of information swapping had a future beyond piggybacking on the success of Napster.

Thus was born InfraSearch.

The search tool works in much the same way as the earlier Gnutella software. Sites that want to be part of the network can run a piece of software inside their databases, sharing up-to-the-minute search results in the place of MP3 libraries, returning their information to anyone who asks for it through the InfraSearch site.

If, for example, a person were to query InfraSearch about "Gnutella," and CNET News.com were running the search agent in its database, the search engine could immediately turn up all of the stories about Gnutella that had been written to the moment by News.com.

Spencer Kimball Developer of InfraSearch   Discussing how the search engine works.

That's different from traditional search engines, which have electronic "spiders" scouring the Web for content and which store those results in a massive searchable database. This model often keeps ordinary search engines days, weeks or months behind, as they strain to keep up with constantly changing Web content.

Traditional search engines also have problems returning "dynamic content"--pages created on the fly by e-commerce or other Web companies as a result of visitor searches. But the Gnutella software could find a link to a page describing a specific Dell Computer configuration created specifically for that search, for example, as Dell itself would be returning the results. Traditional search engines couldn't point to a page like this, because they're limited to static content.

The model also could put far more power behind Web sites, allowing them to determine exactly what they want to be returned to the search engine, InfraSearch's creators say.

This is the feature that puts a new spin on the old search business, doing "for search what the Internet did for communications," Andreessen noted.

"Most of what we've been doing on the Web for the last five or six years has been pretty centralized. But that's not the way the Net itself works," Andreessen said. "It's ironic that it's taken this long to happen."

The site itself is barely live, returning results from just a few other Web sites and without any of the organization used by big search engines. It's just meant as a demonstration project, not as a commercial, Web-wide search, the Gnutella programmers said.

Kan and his associates aren't yet trying to sell the technology, Kan said. They don't have a company in place, nor do they have solid plans to commercialize the InfraSearch idea. But they may move in this direction if it prompts interest from the marketplace, Kan said.

And they're convinced the interest will be there, whether the technology is developed by them or someone else.

"This whole distributed real-time search technology domain is something that's going to change the Internet," Kan said. "This is a whole new technological frontier, ripe for exploitation."

TalkBack

No discussion exists, click here to start it.