Neural models for full text search wsdm2017

Neural Models for
Full Text Search
Could the improvements add up?
Nick Craswell
Microsoft Bing, USA
With: Bhaskar Mitra, Fernando Diaz, Eric Nalisnick, Rich Caruana

“Improvements that don’t add up”
Controversial paper finding poor/no gains since 1998
In a very specific IR task: Average precision, document ranking
“Our longitudinal analysis of published IR results in SIGIR and CIKM
proceedings from 1998-2008 has uncovered the fact that adhoc retrieval
is not measurably improving.”
Timothy G. Armstrong, Alistair Moffat, William Webber, and Justin Zobel.
Improvements that don't add up: ad-hoc retrieval results since 1998.
CIKM 2009

Real improvements
Pre 1990s: IR fundamentals TF-IDF
1990s: TREC ad hoc
2000s to today: Learning to rank
Learning to rank: Real improvements
by incorporating more features.
The “don’t add up claim” is about
improving on the 1990s models such
as BM25, which are still used as
features today.
Tie-Yan Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval 3, no. 3 (2009): 225-331.
Learning to rank with a variety of
features: Links, document structure,
URL structure, usage data, social data

Improvements in the 1990s
Chris Buckley, Mandar Mitra, Janet A. Walz, and Claire Cardie. "SMART high precision: TREC 7." NIST Special Publication 500-242 TREC-7 (1999)
0
0.1
0.2
0.3
0.4
0.5
0.6
TREC-1
Task
TREC-2
Task
TREC-3
Task
TREC-4
Task
TREC-5
Task
TREC-6
Task TD
TREC-6
Task D
TREC-7
Task
AveragePrecision
TREC-1 SMART
TREC-2 SMART
TREC-3 SMART
TREC-4 SMART
TREC-5 SMART
TREC-6 SMART
TREC-7 SMART

Early TREC ad hoc
<top>
<num> Number: 419
<title> recycle, automobile tires
<desc> Description:
What new uses have been developed
for old automobile tires as a means
of tire recycling?
<narr> Narrative:
A relevant document must show
advantageous uses of recycled tires,
such as: destructive distillation of
scrap rubber for valuable chemicals,
reef building for fish habitats,
filler or binder in asphalt roadway
mixes, and burning in a controlled
environment for heat generation.
</top>
Test set: 528155
419 irrel: 1654
419 rel: 16
419 0 FBIS3-24648 1
419 0 FBIS3-42464 1
419 0 FBIS3-43002 1
419 0 FBIS3-43040 1
419 0 FT921-5259 1
419 0 FT921-667 1
419 0 FT923-438 1
419 0 FT934-13811 1
419 0 FT934-9592 1
419 0 FT941-17541 1
419 0 FT941-5694 1
419 0 FT944-18126 1
419 0 LA040190-0174 1
419 0 LA040890-0156 1
419 0 LA112989-0123 1
419 0 LA121389-0085 1

<DOC> <DOCNO> LA040190-0174 </DOCNO> <DOCID> 198113 </DOCID> <DATE> April 1, 1990, Sunday, Home Edition </DATE> <SECTION> Part A; Page 1; Column 1; National Desk </SECTION> <LENGTH> 2434 words </LENGTH> <HEADLINE> 
COLUMN ONE; BEYOND THE END OF THE ROAD; OLD TIRES HAVE LONG BEEN AN ENVIRONMENTAL PAIN IN THE NECK. DESPITE SHREDDING, CHOPPING, GRINDING, BURNING, BURYING AND DUMPING, THEY'RE A BIGGER NUISANCE THAN EVER. </HEADLINE>
<BYLINE> By RUDY ABRAMSON, TIMES STAFF WRITER </BYLINE> <DATELINE> WASHINGTON </DATELINE> <TEXT> In his relentless celebration of American folkways, Norman Rockwell transformed derelict automobile tires into little petunia
beds and children's swings -- testament to Yankee thrift and the creativity of ordinary folks who extracted their last nickel's worth. Sadly, nobody else has so deftly dispatched worn-out tires into a useful hereafter. Long a
national nuisance, they are piling up across the country faster than ever. From coast to coast, they are being shredded, chopped, ground, burned, buried, dumped in the ocean and exported to the Third World. They are turned into jogging
tracks, crash barriers, jungle gyms, mud flaps, sandals, doormats and hockey pucks. But, despite all of this and a flurry of regulations from aroused statehouses, less than 4% are recycled. The rest are still accumulating in vermin- and
mosquito-infested heaps, embarrassing environmental-protection authorities and inviting arsonists. They are banned from half of the country's landfills and being shut out of more every week because of their shape. They take up too much
valuable space and trap air, causing them to resurface, as sullen and indestructible as ever. With the Environmental Protection Agency projecting a 50% reduction in landfill space by the turn of the century and 80% by the year 2010,
tires appear headed for universal banishment from the family of ordinary garbage. "The environmental movement doesn't want you to put them in the ground, they don't want you to burn them, they don't want you to put them in the sea and,
presumably, they don't want you to send them into space," lamented Don Wilson, the principal lobbyist for the National Tire Dealers and Retreaders Assn. "But they have to go somewhere." Arriving home in Buffalo, N. Y., one day
recently, John Spagnoli, a regional director of the New York State Department of Environmental Quality, looked out an airplane window and saw a huge truckload being dumped off the end of the runway. In the hierarchy of waste hazards,
tires trail far behind nuclear and toxic wastes that pose direct hazards to land, air and water, but they are politically and physically incendiary. "Tires in themselves are not something that I am terribly concerned about," Spagnoli said, "but
I've got tire piles all over the place, one with six or eight million tires, and within the next year or two or three, I'm going to have a fire, and the whole thing is going to come to a head." It is not an idle concern. New York had a
major tire fire just last year near Albany. In February, environmental catastrophe was averted in Canada when firefighters extinguished a blaze that had burned for 17 days in an illegal 14-million-tire dump near Toronto. The most
disastrous tire fire in history burned itself out at Winchester, Va., in July, 1984, but the mess it made remains. In nine months, the blaze consumed 5 million to 7 million tires, polluted the air in four states and contaminated the
ground water. After state and federal expenditures of $4 million, the EPA still has not settled on a plan for cleaning up the zinc contamination of the soil and the water of a nearby lake and the holding pond that caught the boiling oil. 
 It is expected that a water-treatment plant will have to be constructed on the site before the polluted water now being diverted into the holding pond can be allowed to flow into the scenic Shenandoah River. The list of lesser
disasters, led by fires in Colorado, Wisconsin and Texas, is long. By some estimates, there may be 3 billion scrap automobile tires in legal and illegal stockpiles, with another 250 million being thrown away every year. Eighty-five percent of
them will go into stockpiles, illegal dumps or landfills. Lying about, they attract not only arsonists, rats and garden-variety mosquitoes, but the Asian Tiger Mosquito, a known vector of dengue fever in the Far East and shown in the
laboratory to transmit encephalitis and a number of less dangerous viral infections. Believed to have come into the United States in truck tires imported for recapping, the insect has now been identified in 120 counties in 17 states,
and as far north as Chicago. Because tire piles provide watery breeding sites for all manner of mosquitoes, the arrival of the Asian Tigers produced a wave of pressures to get tires into landfills. The United States now requires that
tire casings brought in from Asia for recapping first must be fumigated, steam cleaned or otherwise heat treated. Although the mosquito remains of concern to public health officials, the national garbage crisis and the shortage of
landfills has abated the rush to dispose of tires in landfills. Statutes regulating disposal had been put on the books in 18 states by the end of last year, with an equal number expected to tackle new legislative proposals in 1990.
 Several states, including California, have attached fees to tires, creating funds for disposal or recycling, and others have put sticker fees on new cars. Requirements have been laid down for stockpilers and for "casing jockeys" who
make a buck by taking old tires off the hands of retailers. In Oregon, a state permit is required by anyone transporting more than 10 tires, and heavy fines have been levied against stockpilers operating without permits or ignoring
fire-safety requirements. Minnesota's Legislature -- the first such body to appropriate money for tire disposal -- banned dumping five years ago and mandated a vigorous effort to clean up existing tire piles, using chipped casings for
fill in highway-construction projects. But with the problem growing worse and occasionally causing friction between states as junk tires cross borders, creating illegal dumps, there is now a move toward federal legislation making tire
manufacturers and importers responsible for the recycling of an increasing proportion of their products. A recycling incentive bill introduced in the House by Rep. Esteban E. Torres (D-La Puente) and in the Senate by Sens. John Heinz
(R-Pa.) and Timothy E. Wirth (D-Colo.) would require tire manufacturers and importers to recycle 5% of their products. The requirement, enforced by the EPA, would be increased 5% each year, reaching 50% by the turn of the century. 
Given governments' lack of success in handling the problem so far, the idea of the legislation now before Congress is to create a market for junk tires and jump-start a recycling industry. Tire manufacturers and importers would either recycle
old casings themselves, putting the material back into the production of new tires, or purchase credits from recycling operations. Contrary to popular notions, scrap tires are not worthless -- on the average, each one contains 2.5
gallons of oil -- and neither is there any dearth of ideas for things to do with them. It's just that there are so many tires -- steel reinforced to survive age, potholes, high speeds, quick stops, heat, cold, low shoulders and hard
curbs. But technology has kept apace. New machines will slice through steel belts like liverwurst, turning rubber into the consistency of crumbs, sand or baby powder. For years, California and a growing list of other states
have been experimenting with crumb rubber for road paving and patching, and advocates of the practice maintain that rubber asphalt mixes will outlast conventional asphalt and perhaps provide better traction. Paving a mile with a rubber asphalt
mix can use the equivalent of 16,000 junk tires. Appealing as it sounds, the recycling of old tires into new roads has been a long time coming because the mix is more expensive than ordinary asphalt, because patent issues are involved,
and because the Federal Highway Administration is still at work on standards that specify ratios and performance requirements. The story is similar across the recycling front. Retreading has been thrown into steady decline
with the arrival of inexpensive new tires from the Far East, making new tires less expensive than retreads. Although shops across the country have the capacity to put new treads on as many as 25 million tires a year, only an estimated 12.5
million passenger-car tires were retreaded last year. In the mid '80s, 12% of the more than 200 million auto tires were being retreaded. By last year, the number had declined to 7.5%. In spite of the technology to retread or
reincarnate bald and near-bald casings into anything from dish drains to carbon black, it is more profitable to ship worn tires to Third World countries for continued service than it is to recycle them. But the feeble waste-tire market
has never discouraged eccentrics and dreamers, who have accumulated mountains of them with an eye to the day when sky-high energy prices would make them rich. One such collector may have altered the course of the waste-tire crisis.
 Over a period of 20 years, Ed Filben collected the biggest pile of junk tires in the history of the world, 30 million to 40 million of them filling a whole valley in the hills outside of Modesto. On a slope above the
stupefying dump, Oxford Energy Inc., a New York-based company, is burning tires from Filben's collection in a $41-million high-tech plant. The Oxford plant, which obtained the tires in a lease arrangement with Filben, generates 14 megawatts of
electricity, sufficient to supply the needs of 15,000 homes. The electricity is sold to PG&E, the local utility. With its 2,000-degree incinerator going 24 hours a day and consuming 4 million tires a year, the pile could disappear
by about the turn of the century. But meanwhile, the company is collecting more tires -- those of more recent vintage produce more energy -- and expects to continue burning after the massive Filben collection is gone. Environmental
groups, long alarmed over the presence of the outrageous dump, have opposed the import of any more tires, but the plant has complied with state air quality requirements and received environmental citations from the state and the U.S. Department
of Energy. Moving into the black after losing money for half of its first two years, it is being shown off to visitors from other towns where the Oxford company wants to start up new operations. "When we started, we were thinking in
terms of 20 or 40 plants around the country," said Oxford Vice President Robert Graulich, "but we have decided that is neither possible nor desirable." The success at Modesto has not made expansion easy. If abandoned tires epitomize
decay, the notion of tire burning conjures the smell of rubber and the image of oily black smoke. When Oxford proposed to collect millions of tires from across New England and burn them in a tire-to-electricity plant at Stirling,
Conn., it created an uproar across the nearby border in Rhode Island. Concerned that the operation would foul the state's largest water reservoir, the Rhode Island Legislature adopted a bill to block tires from being shipped out of the state to
an incinerating facility within 200 miles of its state line. But after four years, construction is beginning. Oxford is collecting and chipping tires from across New England. According to Graulich, it expects to be collecting them at a
rate of 11 million a year by the time it goes into operation. Meanwhile, Oxford has set its sights on Lackawanna, N. Y., an old Lake Erie steel town, for a third plant, provoking a debate that stands to become a metaphor for the
national waste-disposal crisis and the struggle against tire piles. Lackawanna is financially strapped, devastated by the shutdown of the venerable Bethlehem Steel Co. plant. A monumental cleanup task remains at the abandoned site,
while Bethlehem's only remaining operation, a coke oven, continues to regularly belch pollution, bringing it fines from the state Department of Environmental Conservation. It is the epitome of a Rust Belt town trying to make a
transition to a new economy. It desperately needs new revenue. Oxford Energy's proposal to locate a tire-fed electric power plant twice the size of the Modesto facility produced a sharp split in the town, although the operation would
pay it $980,000 to start up and about $550,000 per year in lieu of taxes. The problem is that Lackawanna is in an area of the state that has excess landfill, two hazardous waste facilities and a nuclear waste dump. Before Oxford
approached with its proposal for a tire-burning plant, the giant waste disposal firm Browning-Ferris Industries made Lackawanna its choice as a location for a huge medical waste processing operation. "The aura of waste has overpowered
rational discussion," said Spagnoli, the state environmental official whose region includes Lackawanna. "There is a garbage phobia because there isn't a hazardous waste made that we don't handle in western New York." At Oxford,
Graulich professes not to be surprised that "we got swept up in all the unfounded fears. We knew that what we were doing was unusual and that we would always have to go through a protracted public education process." Other communities
have already been presented with the issues now facing Lackawanna. When a Utah-based company proposed to build a plant to generate electricity from shredded tires at Rialto near San Bernardino, a complex legal battle ensued, involving
several counties and municipalities and pitting the South Coast Air Quality Management District against its own appeals board. The fight has gone on for more than five years with two trials and two trips to an appeal court, and the opposition
thus far prevailing. Another company, Resource Technology Inc., has hopes of turning old tires into profit at a prototype tire-burning plant at Claxton, Ga. Rather than generating electricity, the plan in Florida is to make a profit by
recovering oil, zinc and carbon black, among other byproducts. The firm plans to apply for a permit for a full-scale facility in Polk County, Fla., where there is a tire pile estimated at 10 million, one of a half-dozen in the state with 5
million casings or more. In Congress, in the tire business, and among environmental groups there remain doubts that incineration or current recycling is the answer to the tire problem. There is a notion that there is something
to be learned from the junk dealers who have created the eyesores and fire hazards with their massive collections. Shredded tires, it is suggested, might be put into lined landfills, and covered with another liner beneath a layer of
dirt. Immune to decomposition, they would remain an energy bank against the day when oil prices might make it commercially attractive to recover them. </TEXT> <GRAPHIC> Photo, In Retirement: Old tires have long been an environmental
nuisance across the United States and the problem is not getting better. From coast to coast they are shredded, chopped, ground, buried, burned, dumped, even recycled into other products. But, as the Column One shows, worn-out tires are piling
up faster than ever. At right, a worker stacks discarded tires at a plant in Westley, Calif., where they will later be burned. ; Photo, (Bulldog Edition) Cat surveys tires at Modesto Energy Project of the Oxford Energy Co. ; Photo, (Bulldog
Edition) Workers stack discarded tires at plant in Westley, Calif., where they are burned in an incinerator. STEVE DYKES / Los Angeles Times </GRAPHIC> <TYPE> Non Dup </TYPE> <SUBJECT> OXFORD ENERGY CO; TIRES; RUBBER; WASTE
MANAGEMENT; WASTE TO ENERGY PLANTS; RUBBISH COLLECTION; RECYCLING; ENVIRONMENT; POLLUTION </SUBJECT> </DOC>
recycle, automobile tires
What new uses have been developed for old
automobile tires as a means of tire
recycling?

April 1, 1990, Sunday, Home Edition
BEYOND THE END OF THE ROAD
OLD TIRES HAVE LONG BEEN AN ENVIRONMENTAL PAIN IN THE NECK.
DESPITE SHREDDING, CHOPPING, GRINDING, BURNING, BURYING AND
DUMPING, THEY'RE A BIGGER NUISANCE THAN EVER
By RUDY ABRAMSON, TIMES STAFF WRITER
WASHINGTON
In his relentless celebration of American folkways, Norman
Rockwell transformed derelict automobile tires into little petunia
beds and children's swings -- testament to Yankee thrift and the
creativity of ordinary folks who extracted their last nickel's
worth.
Sadly, nobody else has so deftly dispatched worn-out tires into a
useful hereafter.
Long a national nuisance, they are piling up across the country
faster than ever.
From coast to coast, they are being shredded, chopped, ground,
burned, buried, dumped in the ocean and exported to the Third
World. They are turned into jogging tracks, crash barriers, jungle
gyms, mud flaps, sandals, doormats and hockey pucks. But, despite
all of this and a flurry of regulations from aroused statehouses,
less than 4% are recycled. The rest are still accumulating in
vermin- and mosquito-infested heaps, embarrassing environmental-
protection authorities and inviting arsonists.
They are banned from half of the country's landfills and being
shut out of more every week because of their shape. They take up
too much valuable space and trap air, causing them to resurface,
as sullen and indestructible as ever.
With the Environmental Protection Agency projecting a 50%
reduction in landfill space by the turn of the century and 80% by
the year 2010, tires appear headed for universal banishment from
the family of ordinary garbage.
"The environmental movement doesn't want you to put them in the
ground, they don't want you to burn them, they don't want you to
put them in the sea and, presumably, they don't want you to send
them into space," lamented Don Wilson, the principal lobbyist for
the National Tire Dealers and Retreaders Assn. "But they have to
go somewhere."
Arriving home in Buffalo, N. Y., one day recently, John Spagnoli,
a regional director of the New York State Department of
Environmental Quality, looked out an airplane window and saw a
huge truckload being dumped off the end of the runway.
In the hierarchy of waste hazards, tires trail far behind nuclear
and toxic wastes that pose direct hazards to land, air and water,
but they are politically and physically incendiary. "Tires in
themselves are not something that I am terribly concerned about,"
Spagnoli said, "but I've got tire piles all over the place, one
with six or eight million tires, and within the next year or two
or three, I'm going to have a fire, and the whole thing is going
to come to a head."
It is not an idle concern. New York had a major tire fire just
last year near Albany. In February, environmental catastrophe was
averted in Canada when firefighters extinguished a blaze that had
burned for 17 days in an illegal 14-million-tire dump near
Toronto.
The most disastrous tire fire in history burned itself out at
Winchester, Va., in July, 1984, but the mess it made remains.
In nine months, the blaze consumed 5 million to 7 million tires,
polluted the air in four states and contaminated the ground water.
After state and federal expenditures of $4 million, the EPA still
has not settled on a plan for cleaning up the zinc contamination
of the soil and the water of a nearby lake and the holding pond
that caught the boiling oil.
It is expected that a water-treatment plant will have to be
constructed on the site before the polluted water now being
diverted into the holding pond can be allowed to flow into the
scenic Shenandoah River.
The list of lesser disasters, led by fires in Colorado, Wisconsin
and Texas, is long. By some estimates, there may be 3 billion
scrap automobile tires in legal and illegal stockpiles, with
another 250 million being thrown away every year. Eighty-five
percent of them will go into stockpiles, illegal dumps or
landfills.
Lying about, they attract not only arsonists, rats and garden-
variety mosquitoes, but the Asian Tiger Mosquito, a known vector
of dengue fever in the Far East and shown in the laboratory to
transmit encephalitis and a number of less dangerous viral
infections.
Believed to have come into the United States in truck tires
imported for recapping, the insect has now been identified in 120
counties in 17 states, and as far north as Chicago.
Because tire piles provide watery breeding sites for all manner of
mosquitoes, the arrival of the Asian Tigers produced a wave of
pressures to get tires into landfills. The United States now
requires that tire casings brought in from Asia for recapping
first must be fumigated, steam cleaned or otherwise heat treated.
Although the mosquito remains of concern to public health
officials, the national garbage crisis and the shortage of
landfills has abated the rush to dispose of tires in landfills.
Statutes regulating disposal had been put on the books in 18
states by the end of last year, with an equal number expected to
tackle new legislative proposals in 1990.
Several states, including California, have attached fees to tires,
creating funds for disposal or recycling, and others have put
sticker fees on new cars. Requirements have been laid down for
stockpilers and for "casing jockeys" who make a buck by taking old
tires off the hands of retailers.
In Oregon, a state permit is required by anyone transporting more
than 10 tires, and heavy fines have been levied against
stockpilers operating without permits or ignoring fire-safety
requirements.
Minnesota's Legislature -- the first such body to appropriate
money for tire disposal -- banned dumping five years ago and
mandated a vigorous effort to clean up existing tire piles, using
chipped casings for fill in highway-construction projects.
But with the problem growing worse and occasionally causing
friction between states as junk tires cross borders, creating
illegal dumps, there is now a move toward federal legislation
making tire manufacturers and importers responsible for the
recycling of an increasing proportion of their products.
A recycling incentive bill introduced in the House by Rep. Esteban
E. Torres (D-La Puente) and in the Senate by Sens. John Heinz (R-
Pa.) and Timothy E. Wirth (D-Colo.) would require tire
manufacturers and importers to recycle 5% of their products. The
requirement, enforced by the EPA, would be increased 5% each year,
reaching 50% by the turn of the century.
Given governments' lack of success in handling the problem so far,
the idea of the legislation now before Congress is to create a
market for junk tires and jump-start a recycling industry. Tire
manufacturers and importers would either recycle old casings
themselves, putting the material back into the production of new
tires, or purchase credits from recycling operations.
Contrary to popular notions, scrap tires are not worthless -- on
the average, each one contains 2.5 gallons of oil -- and neither
is there any dearth of ideas for things to do with them.
It's just that there are so many tires -- steel reinforced to
survive age, potholes, high speeds, quick stops, heat, cold, low
shoulders and hard curbs.
But technology has kept apace. New machines will slice through
steel belts like liverwurst, turning rubber into the consistency
of crumbs, sand or baby powder.
For years, California and a growing list of other states have been
experimenting with crumb rubber for road paving and patching, and
advocates of the practice maintain that rubber asphalt mixes will
outlast conventional asphalt and perhaps provide better traction.
Paving a mile with a rubber asphalt mix can use the equivalent of
16,000 junk tires.
Appealing as it sounds, the recycling of old tires into new roads
has been a long time coming because the mix is more expensive than
ordinary asphalt, because patent issues are involved, and because
the Federal Highway Administration is still at work on standards
that specify ratios and performance requirements.
The story is similar across the recycling front.
Retreading has been thrown into steady decline with the arrival of
inexpensive new tires from the Far East, making new tires less
expensive than retreads. Although shops across the country have
the capacity to put new treads on as many as 25 million tires a
year, only an estimated 12.5 million passenger-car tires were
retreaded last year. In the mid '80s, 12% of the more than 200
million auto tires were being retreaded. By last year, the number
had declined to 7.5%.
In spite of the technology to retread or reincarnate bald and
near-bald casings into anything from dish drains to carbon black,
it is more profitable to ship worn tires to Third World countries
for continued service than it is to recycle them.
But the feeble waste-tire market has never discouraged eccentrics
and dreamers, who have accumulated mountains of them with an eye
to the day when sky-high energy prices would make them rich.
One such collector may have altered the course of the waste-tire
crisis.
Over a period of 20 years, Ed Filben collected the biggest pile of
junk tires in the history of the world, 30 million to 40 million
of them filling a whole valley in the hills outside of Modesto.
On a slope above the stupefying dump, Oxford Energy Inc., a New
York-based company, is burning tires from Filben's collection in a
$41-million high-tech plant. The Oxford plant, which obtained the
tires in a lease arrangement with Filben, generates 14 megawatts
of electricity, sufficient to supply the needs of 15,000 homes.
The electricity is sold to PG&E, the local utility.
With its 2,000-degree incinerator going 24 hours a day and
consuming 4 million tires a year, the pile could disappear by
about the turn of the century. But meanwhile, the company is
collecting more tires -- those of more recent vintage produce more
energy -- and expects to continue burning after the massive Filben
collection is gone.
Environmental groups, long alarmed over the presence of the
outrageous dump, have opposed the import of any more tires, but
the plant has complied with state air quality requirements and
received environmental citations from the state and the U.S.
Department of Energy. Moving into the black after losing money for
half of its first two years, it is being shown off to visitors
from other towns where the Oxford company wants to start up new
operations.
"When we started, we were thinking in terms of 20 or 40 plants
around the country," said Oxford Vice President Robert Graulich,
"but we have decided that is neither possible nor desirable."
The success at Modesto has not made expansion easy. If abandoned
tires epitomize decay, the notion of tire burning conjures the
smell of rubber and the image of oily black smoke.
When Oxford proposed to collect millions of tires from across New
England and burn them in a tire-to-electricity plant at Stirling,
Conn., it created an uproar across the nearby border in Rhode
Island. Concerned that the operation would foul the state's
largest water reservoir, the Rhode Island Legislature adopted a
bill to block tires from being shipped out of the state to an
incinerating facility within 200 miles of its state line.
But after four years, construction is beginning. Oxford is
collecting and chipping tires from across New England. According
to Graulich, it expects to be collecting them at a rate of 11
million a year by the time it goes into operation.
Meanwhile, Oxford has set its sights on Lackawanna, N. Y., an old
Lake Erie steel town, for a third plant, provoking a debate that
stands to become a metaphor for the national waste-disposal crisis
and the struggle against tire piles.
Lackawanna is financially strapped, devastated by the shutdown of
the venerable Bethlehem Steel Co. plant. A monumental cleanup task
remains at the abandoned site, while Bethlehem's only remaining
operation, a coke oven, continues to regularly belch pollution,
bringing it fines from the state Department of Environmental
Conservation.
It is the epitome of a Rust Belt town trying to make a transition
to a new economy. It desperately needs new revenue.
Oxford Energy's proposal to locate a tire-fed electric power plant
twice the size of the Modesto facility produced a sharp split in
the town, although the operation would pay it $980,000 to start up
and about $550,000 per year in lieu of taxes.
The problem is that Lackawanna is in an area of the state that has
excess landfill, two hazardous waste facilities and a nuclear
waste dump. Before Oxford approached with its proposal for a tire-
burning plant, the giant waste disposal firm Browning-Ferris
Industries made Lackawanna its choice as a location for a huge
medical waste processing operation.
"The aura of waste has overpowered rational discussion," said
Spagnoli, the state environmental official whose region includes
Lackawanna. "There is a garbage phobia because there isn't a
hazardous waste made that we don't handle in western New York."
At Oxford, Graulich professes not to be surprised that "we got
swept up in all the unfounded fears. We knew that what we were
doing was unusual and that we would always have to go through a
protracted public education process."
Other communities have already been presented with the issues now
facing Lackawanna.
When a Utah-based company proposed to build a plant to generate
electricity from shredded tires at Rialto near San Bernardino, a
complex legal battle ensued, involving several counties and
municipalities and pitting the South Coast Air Quality Management
District against its own appeals board. The fight has gone on for
more than five years with two trials and two trips to an appeal
court, and the opposition thus far prevailing.
Another company, Resource Technology Inc., has hopes of turning
old tires into profit at a prototype tire-burning plant at
Claxton, Ga. Rather than generating electricity, the plan in
Florida is to make a profit by recovering oil, zinc and carbon
black, among other byproducts. The firm plans to apply for a
permit for a full-scale facility in Polk County, Fla., where there
is a tire pile estimated at 10 million, one of a half-dozen in the
state with 5 million casings or more.
In Congress, in the tire business, and among environmental groups
there remain doubts that incineration or current recycling is the
answer to the tire problem.
There is a notion that there is something to be learned from the
junk dealers who have created the eyesores and fire hazards with
their massive collections.
Shredded tires, it is suggested, might be put into lined
landfills, and covered with another liner beneath a layer of dirt.
Immune to decomposition, they would remain an energy bank against
the day when oil prices might make it commercially attractive to
recover them.
Photo, In Retirement: Old tires have long been an environmental
nuisance across the United States and the problem is not getting
better. From coast to coast they are shredded, chopped, ground,
buried, burned, dumped, even recycled into other products. But, as
the Column One shows, worn-out tires are piling up faster than
ever. At right, a worker stacks discarded tires at a plant in
Westley, Calif., where they will later be burned. ; Photo,
(Bulldog Edition) Cat surveys tires at Modesto Energy Project of
the Oxford Energy Co. ; Photo, (Bulldog Edition) Workers stack
discarded tires at plant in Westley, Calif., where they are burned
in an incinerator. STEVE DYKES / Los Angeles Times
recycling?
Full text search task:
• Fundamental
• Pure

Overview of the Eighth Text REtrieval Conference (TREC-8)
Ellen M. Voorhees, Donna Harman (NIST)
[Regarding top ad hoc runs] The
immediate conclusion to be drawn
from Figure 6 is that there are many
approaches that lead to essentially
the same retrieval effectiveness.
Yet while there are differences in the
details of these approaches, they all
share two properties that we can
therefore conclude are fundamental
to effective retrieval performance. Of
primary importance is the use of a
high-quality weighting scheme.
Query expansion using terms from
highly-ranked documents or
documents related to highly-ranked
documents is also beneficial.

160 the
80 of
72 and
71 a
69 tire
68 to
56 in
33 is
26 for
24 it
21 state
20 that
19 with
19 into
19 have
18 year
18 from
18 be
17 new
17 million
17 by
16 thei
16 on
16 an
15 ha
15 at
14 them
14 plant
14 but
14 burn
14 ar
13 recycl
13 or
12 environment
11 wast
11 than
11 oxford
11 more
11 dump
10 there
10 s
10 been
10 as
9 pile
8 would
8 we
8 up
8 requir
8 oper
8 not
8 landfil
8 energi
8 collect
7 put
7 old
7 make
7 across
7 5
6 when
6 water
6 turn
6 t
6 retread
6 remain
6 out
6 other
6 now
6 long
6 lackawanna
6 go
6 fire
6 far
6 electr
6 dispos
6 case
6 becaus
5 will
5 want
5 two
5 their
5 shred
5 rubber
5 propos
5 problem
5 mosquito
5 last
5 junk
5 import
5 hazard
5 don
5 dai
5 creat
5 countri
5 compani
5 coast
5 air
5 000
4 you
4 york
4 who
4 where
4 were
4 town
4 time
4 stockpil
4 steel
4 start
4 product
4 oil
4 near
4 nation
4 modesto
4 made
4 inciner
4 illeg
4 had
4 ground
4 filben
4 facil
4 expect
4 ever
4 concern
4 black
4 asphalt
4 all
4 against
4 after
4 about
3 worn
3 world
3 us
3 unit
3 third
3 technolog
3 still
3 spagnoli
3 space
3 so
3 site
3 said
3 road
3 qualiti
3 project
3 profit
3 produc
3 pollut
3 plan
3 photo
3 permit
3 over
3 ordinari
3 onli
3 off
3 number
3 nuisanc
3 notion
3 mix
3 manufactur
3 less
3 legisl
3 i
3 home
3 high
3 half
3 graulich
3 gener
3 garbag
3 five
3 first
3 feder
3 estim
3 end
3 edit
3 e
3 depart
3 crisi
3 counti
3 continu
3 construct
3 come
3 clean
3 chop
3 centuri
3 buri
3 befor
3 arriv
3 appeal
3 anoth
3 although
3 4
2 zinc
2 y
2 worker
2 within
2 whole
2 westlei
2 wa
2 tiger
2 thrown
2 through
2 thing
2 themselv
2 take
2 such
2 success
2 stack
2 someth
2 shown
2 ship
2 sever
2 scrap
2 rhode
2 regul
2 region
2 recov
2 recent
2 recap
2 public
2 provid
2 protect
2 process
2 price
2 pond
2 pave
2 offici
2 nuclear
2 neither
2 need
2 nearbi
2 n
2 move
2 monei
2 mile
2 might
2 meanwhil
2 massiv
2 market
2 mani
2 mai
2 locat
2 list
2 line
2 legislatur
2 legal
2 lake
2 just
2 john
2 issu
2 island
2 involv
2 industri
2 increas
2 includ
2 inc
2 idea
2 huge
2 hold
2 histori
2 highwai
2 heat
2 head
2 handl
2 grow
2 group
2 got
2 gone
2 get
2 four
2 firm
2 fine
2 fill
2 fee
2 faster
2 expens
2 everi
2 epitom
2 epa
2 england
2 east
2 each
2 do
2 discard
2 despit
2 declin
2 dealer
2 d
2 crumb
2 contamin
2 consum
2 congress
2 co
2 chip
2 caus
2 carbon
2 car
2 can
2 california
2 calif
2 bulldog
2 border
2 blaze
2 bill
2 better
2 bethlehem
2 belt
2 base
2 ban
2 bald
2 automobil
2 attract
2 asian
2 arsonist
2 ani
2 among
2 accumul
2 abandon
2 80
2 7
2 50
2 40
2 200
2 20
2 2
2 1990
2 17
2 14
2 12
2 10
recycl automobil tire
what new us have been develop for old
automobil tire as a mean of tire
recycl

Not Latent Semantic Indexing (LSI)?
Not Latent Dirichlet Allocation (LDA)?
• LSI was in early TREC
• LDA more recent
• Not needed for long text?
• Doesn’t work?
• Doesn’t always work?

Norman Rockwell transformed derelict automobile
tires into little petunia beds and children's
swings
From coast to coast, they are being shredded,
chopped, ground, burned, buried, dumped in the
ocean and exported to the Third World.
They are turned into jogging tracks, crash
barriers, jungle gyms, mud flaps, sandals,
doormats and hockey pucks.
Minnesota's Legislature ... using chipped casings
for fill in highway-construction projects.
New machines will slice through steel belts like
liverwurst, turning rubber into the consistency
of crumbs, sand or baby powder.
states have been experimenting with crumb rubber
for road paving and patching
it is more profitable to ship worn tires to
Third World countries for continued service than
it is to recycle them.
Utah-based company proposed to build a plant to
generate electricity from shredded tires at
Rialto near San Bernardino
Resource Technology Inc., has hopes of turning
old tires into profit at a prototype tire-
burning plant at Claxton, Ga
Rather than generating electricity, the plan in
Florida is to make a profit by recovering oil,
zinc and carbon black, among other byproducts.
Not passage-level evidence?
recycling?

DNN vs 1990s IR
Artist’s impression of total victory
0
0.1
0.2
0.3
0.4
0.5
0.6
TREC-1
Task
TREC-2
Task
TREC-3
Task
TREC-4
Task
TREC-5
Task
TREC-6
Task TD
TREC-6
Task D
TREC-7
Task
Blind
Test
AveragePrecision
TREC-1 SMART
TREC-2 SMART
TREC-3 SMART
TREC-4 SMART
TREC-5 SMART
TREC-6 SMART
TREC-7 SMART
TREC-26+ DNN

Neural Models for Full Text Search
• Problem: Full text search
• Concerns:
• Soundly beat TREC-8 methods: a) old tests, and b) new (blind) tests
• Word matching+weighting? Query expansion?
• Incorporating a latent space?
• Considering passage-level information?
• Exploring the solution space
1. Using word embeddings
2. Using deep neural nets
• Towards a TREC “deep learning track”

Local and distributed representation of data
Local representation
• One unit for “grandmother”
• 0000000000000100000000000
• Brittle under noise
• Precise
Distributed representation
• “grandmother” is a pattern
• 0100011000001
• More robust to noise
• Potentially find “mother” and
“aunt” nearby in vector space
G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. Parallel distributed processing: Explorations in the microstructure of
cognition, vol. 1. chapter 3. Distributed Representations, pages 77–109. MIT Press, Cambridge, MA, USA, 1986

Vector space models of word meaning
Yale appears with: college, university, school, students, campus,
president, student, research, New Haven, colleges, graduate, century,
library, faculty, undergraduate, history, class, Harvard, buildings
Harvard appears with: university, college, students, president, school,
library, undergraduate, student, century, united, graduate, academic,
Cambridge, states, research, Radcliffe, John, universities
Appear with similar words  Harvard and Yale are related
Firth, J. R. A synopsis of linguistic theory 1930–1955. In Studies in Linguistic Analysis, pp. 1–32. Blackwell, Oxford. (1957).
Peter D. Turney and Patrick Pantel. "From frequency to meaning: Vector space models of semantics." Journal of AI Research 37, no. 1 (2010): 141-188.

Neural word embedding
Continuous bag-of-words (word2vec)
• WIN: 300 dimensions, dense
• Nearest neighbors (cosine)
• Yale: Harvard – NYU – Cornell
• Seahawks: 49ers – Broncos
• Eminem: Rihanna – Ludacris
and shortly
after was
renamed
College in
recognition
of a
yale
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. "Distributed representations of words and phrases and their compositionality." NIPS 2013
|V|
300
|V||V|
300
|V|

Idea: Use both WIN and WOUT
• Yale: Harvard – NYU – Cornell – Tulane – Tufts
• Yale: faculty – alumni – orientation – haven – graduate
• Seahawks: 49ers – Broncos – Packers – NFL – Steelers
• Seahawks: highlights – jerseys – tshirts – Seattle – hats
• Eminem: Rihanna – Ludacris – Kanye – Beyonce – 2pac
• Eminem: rap – featuring – tracklist – diss – performs
WIN WOUT
Eric Nalisnick, Bhaskar Mitra, Nick Craswell, and Rich Caruana. "Improving document ranking with dual word embeddings." WWW 2016 Poster.
Dual embeddings

Idea: Compare all Q-D word pairs
Rank according to
Where
“aboutness”
Q-D centroid comparison is
equiv to all-pairs Q-D

Dual Embedding Space Model (DESM)
Full-text reranking test: NDCG@10 on Web data
Labels Click labels
BM25 44.77 49.26
LSA 44.24* 52.05*
DESM (IN-IN, trained on body text) 45.51* 53.32*
DESM (IN-IN, trained on queries) 46.36* 54.20*
DESM (IN-OUT, trained on body text) 46.57* 54.13*
DESM (IN-OUT, trained on queries) 47.89* 55.84*
* Different from BM25 (p<0.05)

Bhaskar Mitra, Eric Nalisnick, Nick Craswell, and Rich Caruana. "A dual embedding space model for document ranking." arXiv:1602.01137 (2016).
IN-OUT and BM25 make
different mistakes

Get the data
IN+OUT Embeddings for 2.7M words
trained on 600M+ Bing queries
research.microsoft.com/projects/DESM
Download

Idea: Try per-query embeddings
Learn word embeddings that
are “local” to the current
query
The word apple will have
different associations in
• Global: Train on all docs
• Local: Train on docs from
Q=apple pie recipes
Covariate shift
or
Fernando Diaz, Bhaskar Mitra, and Nick Craswell. "Query Expansion with Locally-Trained Word Embeddings." ACL 2016

Nearest neighbors of “cut”
Global Local (Q=gasoline tax)
cutting tax
squeeze deficit
reduce vote
slash budget
reduction reduction
spend house
lower bill
halve plan
soften spend
 Evaluation: Query
expansion in Indri.

Results

Visualizing a query
Query (blue) ‘ocean remote sensing’
Good expansion terms (red)
Other candidates (white)
Using t-SNE.
global
local
Learn per-query? Per-corpus?
Per-task?

http://fortune.com/ai-artificial-intelligence-deep-machine-learning/
People call out:
• Datasets
• GPUs
• Some NN advances
Also: An open approach

Deep Learning: Representation learning with
multiple layers and multiple levels of abstraction
LeCun, Y., Bengio, Y. and Hinton, G. E. “Deep Learning.” Nature, Vol. 521 (2015) pp 436-444
Selectivity-invariance dilemma

Zeiler and Fergus. Visualizing and understanding convolutional networks. European conference on computer vision 2014
Deconvnet: Map
back to pixel space
Corresponding
image patches
Top-9 activations
each feature map

How full text representation could work
• Input: Word/character sequence (query and doc full body text)
• Task: Document ranking given a query (IR objective)
• Layers: Distributed representation of word/phrase/sequence
meaning (end-to-end optimization of IR task)

DNNs in industry
Baidu WSDM 2015 (short text) Jeff Dean WSDM 2016
http://www.wsdm-conference.org/2016/slides/WSDM2016-Jeff-Dean.pdf
http://www.wsdm-conference.org/2015/wp-content/uploads/2015/02/WSDM-Talk-Baidu-Search.pdf

Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck.
"Learning deep structured semantic models for web search using clickthrough data." CIKM 2013.
Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil.
"A latent semantic model with convolutional-pooling structure for information retrieval." CIKM 2014.
Convolutional
DSSM
• Short text data:
• Pos: Query-ClickedTitle
• Neg: Query-RandomTitle
• 128 dimensional 𝑦 𝑄 and 𝑦 𝐷
• Rank by: cosine(𝑦 𝑄, 𝑦 𝐷)
• Loss function related to
RankNet (like IR objective)

Convolutional
DSSM
300 choose 5 = 19,582,837,560

Recent result
• CDSSM was:
• Document title
• Trained on clicks
• Tested on labels
• Modified CDSSM
• Full text search
• Trained on labels
• Tested on labels
Large DNN training sets were
crucial in vision and speech
Here’s evidence we need that too This and more: Mitra, Diaz and Craswell. Learning to Match Using Local and
Distributed Representations of Text for Web Search. WWW2017

model introduced top-1 err. top-5 err. images/s
resnet-50-dag 2015 24.6 7.7 396.3
resnet-101-dag 2015 23.4 7.0 247.3
resnet-152-dag 2015 23.0 6.7 172.5
matconvnet-vgg-verydeep-16 2014 28.3 9.5 200.9
vgg-verydeep-19 2014 28.7 9.9 166.2
vgg-verydeep-16 2014 28.5 9.9 200.2
googlenet-dag 2014 34.2 12.9 770.6
matconvnet-vgg-s 2013 37.0 15.8 586.2
matconvnet-vgg-m 2013 36.9 15.5 1212.5
matconvnet-vgg-f 2013 41.4 19.1 2482.7
vgg-s 2013 36.7 15.3 560.1
vgg-m 2013 37.3 15.9 1025.1
vgg-f 2013 41.1 18.8 1118.9
vgg-m-128 2013 40.8 18.4 1031.3
vgg-m-1024 2013 37.8 16.1 958.5
vgg-m-2048 2013 37.1 15.8 984.2
matconvnet-alex 2012 41.8 19.2 2133.3
caffe-ref 2012 42.6 19.7 1071.7
caffe-alex 2012 42.6 19.6 1379.8
Pre-trained models
• Particularly from large data
Some ImageNet models available online
http://www.vlfeat.org/matconvnet/pretrained/ 
• Considerations
• Is my dataset smaller (yes)
• How big is my dataset
• How different is it
• Transfer to new task:
• Copy first layers (shallow, deep)
• As frozen weights, or fine-tune
Interesting analysis:
Yosinski, Clune, Bengio, and Lipson.
How transferable are features in deep neural networks?.
NIPS 2014

TREC Deep Learning Track?
• TREC ad hoc document ranking.
Avoid interesting task/structure!
• Maybe NYTimes data?
• Key question: Training data
• Need labels? Labels per-corpus? What
queries? What sampling distribution?
• Key benefit: Blind test in TREC is a
very rigorous test

Neural networks for full text search
• TREC ad hoc: Fundamental IR task
• Ingredients for making progress
• Unsupervised methods for learning IR-appropriate text embeddings
and/or
• Large-scale IR training data and deep neural networks
• Pitfalls: No data. Models aren’t robust/general. Not worth it
• Optimistic view:
• We made fundamental progress on this problem before
• TREC is a great way of testing that
• Ideally reach a new plateau, with many good+equal approaches

Neural models for full text search wsdm2017

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Neural models for full text search wsdm2017

Editor's Notes