Web-Crawling Program ID's Disease Outbreaks

July 18, 2008 -- Scientists are searching within the virtual world and finding real viruses.

Every hour, HealthMap, an infectious disease-tracking Web site, culls through news Web sites, public health list servs, the World Health Organization's online pages, and other Web sites in six different languages to pinpoint outbreaks of disease that real-world doctors can then act on.

"We were originally thinking about how we could expand disease surveillance and pick up outbreaks earlier than traditional methods," said John Brownstein of Harvard Medical School and Children's Hospital Boston, who created HealthMap in September of 2006 with Clark Freifeld, a software developer at Harvard Medical School.

"It was a pilot project, a side gig for us," explained Brownstein.

About nine months ago, HealthMap came to the attention of Google.org, the philanthropic arm of the Internet search giant, which began to fund the team. Then, as Brownstein recalled, "all of a sudden, it just took off."

HealthMap gathers information from the Internet and filters it, removing, for example, duplicated or irrelevant information. It can pinpoint an incident of bubonic plague in Siberia, for example, while ensuring that a "plague" of home foreclosures in northern California doesn't show up on the free access Google Maps.

So far the program can accurately determine the validity of a report about 95 percent of the time, sometimes days before the World Health Organization or the Centers of Disease Control announce disease outbreaks.