By way of decrying a society that left its citizens unbearably restrained, Edith Wharton describes how in New York in the 1870s, women would order dresses from their Paris dressmakers and then leave them in tissue paper at least two years before wearing them in public; the thought of showing them "in advance of the fashion" was unforgivably vulgar. Social life has changed, but cultural life seems just as restricted now – even Animal Collective are held back by trends that seem a couple of years old (and that they helped to invent). When I think back on 2009, I’ll first remember how our impoverished aesthetic generation repeatedly scraped the resin from the cultural trash barrel. Every second person is wearing neon leggings, and the ones who aren’t rock a ‘70s aesthetic, with high-waisted jeans and moccasins. Christmas sweaters are getting impossible to find at the thrift store. Ska revival. Garage rock revival. It never ends.Read More ...
A straight shot west out of Boston on I-90 will carry you, in two hours or less, to Western Massachusetts, where the country still looks like it did twenty or even 40 years ago: college towns, I-91 tracing the same lazy ladder from Springfield up through Holyoke and Northampton, Amherst and Deerfield. Out there it's taken for granted that the houses will be drafty, the winters uniformly long, and that, on any given trip to the local supermarket, one might spot Thurston or Lou or Kim or J, on-and-off locals for more than twenty years. {audio}http://www.archive.org/download/DinosaurJrDrawings/07Drawerings_64kb.mp3{/audio} ... DraweringsRead More ...
The Ex are one of those rare bands that, despite being around for 25 years, have neither gone soft nor stagnated. The 23 tracks on this album all date from their first decade of existence (1980-1990), and if you compare it with recent milestones like Starter Alternator and Turn, you’ll see that while many of the Ex’s virtues are long standing, much has changed. The Ex grew out of Amsterdam’s once-fertile squatters’ subculture, and have always been politically conscious; Singles. Period. includes screeds that oppose American cultural hegemony, Dutch apathy, and eugenics. Their most recent album Turn likewise includes protests against globalization, consumerism, and cultural erosion, but its lyrics are quite nuanced and in touch with the grey areas of the issues when compared with the black and white prescription of 1981’s “Weapons For El Salvador”: .............. {audio}http://www.theex.nl/mp3/The%20Ex%20-%20Trash.mp3{/audio} ... TrashRead More ...
For just over 10 years, London's Guapo has been working in the world of avant and progressive rock. The band's past is a bit hard to track with its numerous lineup changes and guest musicians. The most recent change in roster was the resignation of Matthew Thompson, the founding member of Guapo, which occurred just before the release of 2005's Black Oni. The departure of Thompson has left Guapo with percussionist David Smith and multi-instrumentalist Daniel O'Sullivan. Though O'Sullivan is by no means a founding member of the band, but he was essential in honing the sound on Guapo's last two LPs: Five Suns and Black Oni. These two albums have been pivotal in building Guapo's following of fans, so it's hard not to credit O'Sullivan as an asset to the band.... {audio}http://www.neurotrecordings.com/artists/guapo/audio/Guapo-The%20Selenotrope.mp3 {/audio} ... The SelenotropeRead More ...
A live album can be many things: a candid snapshot, a footnote to a scene, or even just a thrifty alternative to studio time. Antlers, a collection of live Bastro recordings from 1991, is the rarest kind of live album: it illuminates a side of the band that, in turn, casts their previous work in a new light as well.“1991 has been called the year that punk broke. Some of it broke into the mainstream, but some broke into more irregular shards.” David Grubbs’s observation, from the liner notes to Antlers, could also describe the varied musical paths that led from his former band Squirrel Bait to the disparate ’90s groups he and his ex-bandmates went on to found: Slint, Palace Brothers, King Kong, Bitch Magnet, the For Carnation, Tortoise, and of course, Bastro.Read More ...
LIMB is an archival release of some material recorded from 1980-1983, from the early days of Foetus and pre-Foetus. Some of the pieces here have been previously released on the compilation albums. Parts have been released on various single b-sides. Some were excavated from old cassettes and some of it was reconstructed or re-edited from compositions on cassette. One piece is constructed from an organ part written in 1982, which I took the liberty of finishing in 2008. These pieces were made before the introduction of MIDI and sampling technology. {audio}http://brainwashed.com/common/sounds/mp3/foetus-limb-te_deum.mp3{/audio} ....Te DeumRead More ...
Doing justice to Nisht Azoy is a tricky affair, not least because Black Ox Orkestar stand alone, making comparisons and associations nearly impossible. The use of gypsy melodies on the later Firewater albums also comes to mind. Adopting and reinterpreting Jewish diasporic music, Black Ox Orkestar's second album moves through original songs, traditionals and compositions based on folk songs and ballads. The selections are masterful, as is the playing, whether eliciting joy with dance rhythms and singing or delving into slow, sad evocations of melancholy...... {audio}http://www.southern.net/southern/band/BLAOX/sounds/CST29_08_toyte_goyes_in_shineln.mp3{/audio} ....toyte goyes in shinelnRead More ...
To those who detest the comic-bookish faux personas invented by electro musicians, and who think that being cryptic is tantamount to snobbery, the Detroit duo Adult. may be one of the most frustrating bands on the planet. Their paranoiac lyrics and resistance to celebrity can make them difficult to embrace. But their world of anonymous, implied violence is just as humorous as it is vague, and at the end of the night it's meant to be fun. Even as their music remains sterile and distant, artists and people can't help but to seep into their music. As with all the electro artists who "hide their identity," a sense of humor and adventure is helpful in confronting Adult.’s music. Adam Lee Miller and Nicola Kuperus (Adult.) recently answered some interview questions for a six-year old named Lucas, who was apprised of the band's existence by his father.Read More ...
Japan’s Annual Penis Festival – Celebrates Fertility
KOMAKI, Japan — It's springtime in Japan and that means one thing. Actually, two things. Penis festivals and vagina festivals. It may sound like a sophomoric gag. But these are folk rites going back at least 1,500 years, into Japan's agricultural past. They're held to ensure a good harvest and promote baby-making. Maybe they should hold more such festivals. Japan has one of the world's lowest birthrates (1.37 children per woman), which experts blame on stagnant incomes and changing gender relations.Read More ...
All world secret underground bases build for space travelers
The following material comes from people who know the Dulce (underground) base exists. They are people who worked in the labs; abductees taken to the base; people who assisted in the construction; intelligence personal (NSA,CIA,FBI ... ect.) and UFO / inner-earth researchers. This information is meant for those who are seriously interested in the dulce base. for your own protection be advised to “use caution” while investigating this complex.Does a strange world exist beneath our feet? Strange legends have persisted for centuries about the mysterious cavern world and the equally strange beings who inhabit it. More UFOlogists have considered the possibility that UFOs may be emanating from subterranean bases, that UFO aliens have constructed these bases to carry out various missions involving Earth or humans.Read More ...
The "Max Headroom pirating incident" is the fascinating story of the successful hijacking of two television signals in the Chicago, Illinois area on Sunday, November 22, 1987. This feat was accomplished by a mysterious person wearing a Max Headroom mask who somehow over-rode the station signals and then proceeded to perform an illegal broadcast on live television. The incident began when television broadcasts at two different stations (WTTW and WGN) were interrupted by the hijacker on the same day, only hours apart.Read More ...
German-Japanese flight to Moon and Mars in 1945-46
The moon has allways held a significant place for humanity both as a source for romantic inspiration for poets and the like to outstanding curiosity for scientists. Allthough, it is said to be a shadowy place some say of Aliens others say of Top Secret Moon Bases that are supposed to belong to The Third Reich what do you think ? It is said that in the early nineties that Nazies landed on the moon using some sort of giant flying saucer type object. These Nazi flying Saucers were said to stand about 45 mtrs high, had 10 stories of crew quaters and had a diameter of 60 mtrs. Well here is videos and texts that links that story ........Read More ...
These days, with all the pundits preaching doom and the impending collapse of society into some kind of Mad Max style wasteland, it's easy for us to imagine that the economy is as unhealthy as it's ever been. But any historian would give you a hard backhanded smack for even saying that out loud. History is full of economic idiocy, and here are five economic collapses that make 2010 feel like the Renaissance.Read More ...
Black-lip Rattail ............ These sorts of rattails feed in the muddy seafloor by gliding along head down and tail up, powered by gentle undulations of a long fin under the tail. The triangular head has sensory cells underneath that help detect animals buried in the mud or sand. The common name comes from the black edges around the mouth.Read More ...
What Came 'Before' the Big Bang? Leading Physicist Presents a Radical Theory
String theorists Neil Turok of Cambridge University and Paul Steinhardt, Albert Einstein Professor in Science and Director of the Princeton Center for Theoretical Science at Princeton believe that the cosmos we live in was actually created by the cyclical trillion-year collision of two universes (which they define as three-dimensional branes plus time) that were attracted toward each other by the leaking of gravity out of one of the universes. In their view of the universe the complexities of an inflating universe after a Big Bang are replaced by a universe that was already large. flat, and uniform with dark energy as the effect of the other universe constantly leaking gravity into our own and driving its acceleration. According to this theory, the Big Bang was not the beginning of time but the bridge to a past filled with endlessly repeating cycles of evolution, each accompanied by the creation of new matter and the formation of new galaxies, stars, and planets.Read More ...
I work with a bunch of geeks. And that's okay. They do their thing and I do mine. Most of the time I'm happy for them, that they get joy and happiness out of playing with electronics. Admittedly I disagree with a lot of their thoughts about life. People used to believe that the Earth was the center of the universe, then it was the sun, but now we all know that the computer is the focal point of the universe, projecting its cathode ray goodness on our souls. You can't eat, sleep, breathe, live or run a business without one, or so we're told. But if there's one thing I have no tolerance for, it's the geek phenomenon known as slashdot.org, the sorriest case for content on the web I've ever seen pawned off and gleefully accepted by the masses.Read More ...
Learn How to Pronounce the Iceland Volcano Eyjafjallajokull and remember; When He Erupted In 1821, it lasted 2 years
The last time Eyjafjallajökull erupted, it lasted 2 years stretching from 1821-1823. It also erupted in 920 and 1612. Eyjafjallajökull's eruption usually precedes an eruption for another Icelandic volcano called Katla, as it did in 1823. Katla's eruptions are usually more violent than Eyjafjallajökul's. Due to the second activity on Eyjafjallajökull volcano since April 14, there are thousands of flights have been cancelled not only in Europe but also some flights from Asia, America and other continents. More over, it was also reportedly more than ten thousands of air travelers still stranded after a plume of ash cloud spreading across thousands of miles. No need to repeat the same news in every single post, actually there’s an interesting thing from the Iceland volcano’s name Eyjafjallajokull. Pronunciation is so difficult for some of us. Even, many people still don’t know what’s the right pronunciation of Eyjafjallajokull volcano. Did you know that?Read More ...
How drug companies deceive doctors and how hire ghostwriters to produce articles
Following doctor’s orders has become synonymous with danger. Every year, FDA approved drugs kill twice as many people as the total number of U.S. deaths from the Vietnam War. Death by medicine flourishes because deceit, not science, governs a doctor’s prescribing habits. This deceit comes in many forms. Medical ghostwriting and checkbook science are the most prominent.Read More ...
The Marijuana Conspiracy - The Real Reason Hemp isIllegal
MARIJUANAis DANGEROUS. Pot is NOT harmful to the human body or mind.Marijuana does NOT pose a threat to the general public. Marijuana isvery much a danger to the oil companies, alcohol, tobacco industriesand a large number of chemical corporations. Various big businesses,with plenty of dollars and influence, have suppressed the truth from thepeople. Thetruth is if marijuana was utilized for its vast array of commercialproducts, it would create an industrial atomic bomb! Entrepreneurs havenot been educated on the product potential of pot. The super rich haveconspired to spread misinformation about an extremely versatile plantthat, if used properly, would ruin their companies.Read More ...
9/11 has inspired a myriad of memorials who are scattered all across America. Some of them are of questionable taste, others contain strange occult symbolism while others simply piss people off. Here’s the five most offensive.Read More ...
A temple complex in Turkey that predates even the pyramids is rewriting the story of human evolution. They call it potbelly hill, after the soft, round contour of this final lookout in southeastern Turkey. To the north are forested mountains. East of the hill lies the biblical plain of Harran, and to the south is the Syrian border, visible 20 miles away, pointing toward the ancient lands of Mesopotamia and the Fertile Crescent, the region that gave rise to human civilization. And under our feet, according to archeologist Klaus Schmidt, are the stones that mark the spot—the exact spot—where humans began that ascent.Read More ...
The international community has come out in force to condemn and declare war on the Somali fishermen pirates, while discreetly protecting the illegal, unreported and unregulated (IUU) fleets from around the world that have been poaching and dumping toxic waste in Somali waters since the fall of the Somali government eighteen years ago. In 1991, when the government of Somalia collapsed, foreign interests seized the opportunity to begin looting the country’s food supply and using the country’s unguarded waters as a dumping ground for nuclear and other toxic waste.Read More ...
Hindu Nepalis celebrate the ‘great night of Shiva’ smoking hashish and marijuana
KATHMANDU: Narcotic Drugs (Control) Act forbids buying and selling of drugs in the country. The law can slap fines and an imprisonment of up to 20 years if convicted in drug related crimes. But a site at the Pashupatinath Temple area today made a mockery of the law. It was but smoke and mirrors. The holy site of Hindus smoked round-the-clock. The breeze smelled the cannabis as far away as Mitrapark and Gaushala. Some 50,000 Hindu pilgrims from Nepal and India gathered last Saturday (02/13/2010) in Kathmandu’s Pashaupatinath Temple to celebrate Mahashivaratri, the ‘great night of Shiva’. Worshippers, including teenagers, freely bought hashish and marijuana and immersed themselves in the polluted (and potentially infectious) waters of the Bagmati River.Read More ...
Squatting consists of occupying an abandoned or unoccupied space or building, usually residential, that the squatter does not own, rent or otherwise have permission to use. There are one billion squatters globally, that is, about one in every six people on the planet. Yet, according to Kesia Reeve, "squatting is largely absent from policy and academic debate and is rarely conceptualized, as a problem, as a symptom, or as a social or housing movement. In many countries, squatting is in itself a crime; in others, it is only seen as a civil conflict between the owner and the occupants. "Squatters are usually portrayed as worthless scroungers hell-bent on disrupting society." Property law and the state have traditionally favored the property owner. However, in many cases where squatters had de facto ownership, laws have been changed to legitimize their status.Read More ...
The World's First Commercial Brain-Computer Interface + history of BCI
A brain–computer interface (BCI), sometimes called a direct neural interface or a brain–machine interface, is a direct communication pathway between a brain and an external device. BCIs are often aimed at assisting, augmenting or repairing human cognitive or sensory-motor functions. Research on BCIs began in the 1970s at the University of California Los Angeles (UCLA) under a grant from the National Science Foundation, followed by a contract from DARPA. The papers published after this research also mark the first appearance of the expression brain–computer interface in scientific literature.Read More ...
The T2K Experiment - From Tokai To Kamioka - Where is the anti-matter?
From the beginning of 2010, the T2K experiment will fire a beam of muon-neutrinos from Tokai on Japan's east coast, 300km accross the country to a detector at Kamioka. It hopes to investigate the phenomenon of "neutrino oscillations" by looking for "muon neutrinos" oscillating into "electron neutrinos". A million pound detector has been built at the University of Warwick as part of a vital experiment to investigate fundamental particles - neutrinos.Read More ...
I STARTED collecting examples of bizarre experiments years ago while in graduate school studying the history of science. I confess I had no profound intellectual motive; I simply found them fascinating. They filled me with disbelief, astonishment, disgust and - best of all - laughter. With hindsight, perhaps there is a deeper message. These experiments are not the work of cranks.Read More ...
If you're trying to buy happiness, you'd be better off putting your money toward a tropical island get-away than a new computer, a new study suggests. The results show that people's satisfaction with their life-experience purchases — anything from seeing a movie to going on a vacation — tends to start out high and go up over time. On the other hand, although they might be initially happy with that shiny new iPhone or the latest in fashion, their satisfaction with these items wanes with time. The findings, based on eight separate studies, agree with previous research showing that experience-related buys lead to more happiness for the consumer. But the current work provides some insight into why.Read More ...
The giant ALICE detector is already underway at CERN, and researchers are scrambling to add an electromagnetic calorimeter to capture jet-quenching, the newest way to look inside the quark-gluon plasma — the hot, dense state of matter that filled the earliest universe, which the Large Hadron Collider will soon recreate by slamming lead nuclei into one another. CERN's Large Hadron Collider (LHC) is known mainly as the accelerator that will soon begin searching for the Higgs particle, and other new physics, in proton collisions at unprecedented energies — up to 14 TeV (14 trillion electron volts) at the center of mass — and with unprecedented beam intensities. But the same machine will also collide massive nuclei, specifically lead ions, to energies never achieved before in the laboratory.Read More ...
It is not science fiction. Researchers at The Neurosciences Institute in La Jolla have designed a machine that thinks. Today, Darwin 6 consists of a realistically designed simulation of a nervous system housed in a mobile platform called NOMAD (Neurally Organized Mobile Adaptive Device). The research is conducted in the Institute's W.M. Keck Foundation Laboratory of Machine Psychology. Established in 1998 with a grant of 1.5 million from the W.M. Keck Foundation of Los Angeles, the Keck Laboratory studies the neural bases of behavior and how the brain reacts and adapts to a changing world. Its objective is to develop a new generation of powerful models of brain activity. Unlike a robot, NOMAD is an autonomous "being," used as a tool to study how the brain controls behavior. According to neuroscientist Jeffrey Krichmar, Ph.D., NOMAD is at the behavioral level of an infant.Read More ...
Earth's atmosphere came from outer space, find scientists
The gases which formed the Earth's atmosphere - and probably its oceans - did not come from inside the Earth but from outer space, according to a study by University of Manchester and University of Houston scientists. The report published this week in the prestigious international journal 'Science' means that textbook images of ancient Earth with huge volcanoes spewing gas into the atmosphere will have to be rethought.Read More ...
Seven theories of everything that pretend to describe the fundamental nature of the universe
We still don't have a theory that describes the fundamental nature of the universe, but there are plenty of candidates.
The "theory of everything" is one of the most cherished dreams of science. If it is ever discovered, it will describe the workings of the universe at the most fundamental level and thus encompass our entire understanding of nature. It would also answer such enduring puzzles as what dark matter is, the reason time flows in only one direction and how gravity works. Small wonder that Stephen Hawking famously said that such a theory would be "the ultimate triumph of human reason – for then we should know the mind of God". But theologians needn't lose too much sleep just yet. Despite decades of effort, progress has been slow. Rather than one or two rival theories whose merits can be judged against the evidence, there is a profusion of candidates and precious few clues as to which (if any) might turn out to be correct.Read More ...
It was nearly the end ofWWII. At thatsame time, scientist Viktor Schauberger worked on a secret project. Johannes Kepler, whose ideas Schauberger followed, had knowledge of the secretteachings of Pythagoras that had been adopted and kept secret. It wasthe knowledge of Implosion (in this case the utilization of the potential of the inner worlds in the outer world). Hitler knew - as did the Thule and Vril people - that the divine principle was always constructive. A technology however that is based on explosion and therefore is destructive runs against the divine principle. Thus they wanted to create a technology basedon Implosion.Read More ...
Mysterious Radio Waves from Unknown Object in M82 Galaxy
There is something strange is lurking in the galactic neighborhood. An unknown object in galaxy M82 12 million light-years away has started sending out radio waves, and the emission does not look like anything seen anywhere in the universe before except perhaps by Ford Prefect. M82 is starburst galaxy five times as bright as the Milky Way and one hundred times as bright as our galaxy's center. "We don't know what it is," says co-discoverer Tom Muxlow of Jodrell Bank Centre for Astrophysics near Macclesfield, UK. But its apparent sideways velocity is four times the speed of light. This "superluminal" motion occurs usually in high-speed jets of material bursting out by black holes.Read More ...
Black Prince, alien space probe, orbits Earth watching humans
Alexander Kazantsev, a Soviet author of sci-fi books, once said that a mysterious “unaccounted” satellite called Black Prince was spinning around Earth. The writer believed the object might be an alien probe, a messenger from extraterrestrial civilizations. Some people including scientists paid attention to the writer’s hypothesis.U.S. astrophysicist Ronald Bracewell was the first to take the hypothesis seriously. In 1960, he published a study to back his conclusions with data of practical radio engineering.Read More ...
Unsettled Mechanism of Supernova Detonation Gets a New Twist
Type Ia supernovae, often used to calibrate cosmological measurements, may arise from merging white dwarfs, after all
When stellar cataclysms known as type Ia supernovae flare up far across the universe, their brightness and consistency allow astronomers to use them as so-called standard candles to measure cosmological distances. Just over a decade ago, two teams used the supernovae to show that the universe is accelerating in its expansion due to the influence of dark energy, a shocking discovery that thrust type Ia supernovae into the astrophysical limelight. But how exactly did these cosmic mileposts come to be?Read More ...
NASA's WISE Mission Releases Medley of First Images
PASADENA, Calif. -- A diverse cast of cosmic characters is showcased in the first survey images NASA released Wednesday from its Wide-field Infrared Survey Explorer, or WISE. Since WISE began its scan of the entire sky in infrared light on Jan. 14, the space telescope has beamed back more than a quarter of a million raw, infrared images. Four new, processed pictures illustrate a sampling of the mission's targets -- a wispy comet, a bursting star-forming cloud, the grand Andromeda galaxy and a faraway cluster of hundreds of galaxies.Read More ...
Hubble telescope captures image of mysterious x-shaped object in space
Is that a smashed comet or an X-Wing fighter? Scientists are offering up their own theories as to what created the striking star-inspired image, which was captured by NASA's Hubble telescope in January. "Two small and previously unknown asteroids recently collided, creating a shower of debris that is being swept back into a tail from the collision site by the pressure of sunlight," said principal investigator David Jewitt of the University of California at Los Angeles.Read More ...
Strange Images from Space - Photos&videos of the Bizarre in Our Universe
Some weird and unusual objects are floating around in the cosmos. Space is always serving up something new, unusual, and unexpected. Here are images and explanations of obejcts that have amazed and delighted astronomers.Read More ...
Follow Spirit*s Mars progress: ready to "taste" a rock named "Chocolate Hills"
Opportunity at a Sweet Spot on Mars
Guess who has a "sweet tooth?" The Opportunity rover on Mars, of course. The robotic geologist is poised and ready to "taste" a rock named "Chocolate Hills."This rock has a thick, dark-colored coating that is interesting to scientists because many of the rocks in the surrounding area have the same mysterious dark stuff. The coating could be remnants of a layer that was changed by the action of water and weather or, it could be a layer of rock that melted when a meteor (less than a foot across) impacted Mars,ejecting this rock and others and creating the crater "Concepcion".Read More ...
Homeart news Internet Archive - the fastest growing archive of human expression ever created
Internet Archive - the fastest growing archive of human expression ever created
Tucked away in one of the seediest neighborhoods of San Francisco is a roomful of over two hundred computers with a terabyte of data stored on every three. Stairs from the street lead up an intimidating hallway that opens into a room with 15-foot ceilings and just-this-side of hip ductwork in the ceiling. To the right is a storage area with a single desk, to the left are Baker's racks tightly packed with off-the-shelf HP desktop machines, each turned on it's side to maximize the space. Somewhere in all that ductwork, a fan is squeaking painfully. Walking into this echoey, over-warm warehouse space, it's easy to be underwhelmed until you realize what you are looking at: spinning away on these computers is nothing less than a copy of the Internet from 1996 until today.
Hyperbole is easy to generate: over 10 billion pages are held here. The content of a single computer is equivalent to the entire Library of Congress. Over 250 gigabytes of data are added daily. Over 12 terabytes are added every month and there are a total of over 120 terabytes of storage available. As a copy of the entire publicly accessible internet, it is also certainly the worlds largest collection of pornography in a single room.
stairs leading to the Archive
Dim lightsEmbed Embed this video on your site Have you ever wondered how websites used to look back in the day? Wayback Machine lets you find out easily. Oday has done this screencast for us to show all of you some of the many fun things you can do with Wayback MachineThe Internet Archive was founded in 1995 by Brewster Kahle with the intent of preserving what is arguably the fastest growing archive of human expression ever created. The Library of Congress, and other analog equivalents, keep copies of the thousands of books published every year, helping to preserve our paper history, but the amount of content on the internet has grown to dwarf that repository. Also, unlike books on the shelves of libraries, web pages are in a constant state of flux: the average age of a web site is only 19 months with the average page changing every 100 days. The amount of information created and lost every day is staggering. The current state of digital technology and the internet makes it feasible for the Archive to reach it's stated mission of universal access to human knowledge.
The Internet Archive has several locations, the Mission street facility is their co-location site. This no-frills building is where the data is stored. The other primary site is located at the Presidio in the shadow of the Golden Gate Bridge. That San Francisco location is where the most of the staff are and where the day-to-day business of the Archive takes place. There is another site in Egypt that holds a duplicate of the data on the Mission Street hard drives.
Computing Power on a Shoestring budget
A walk through any other co-location site would be a testament to the Cisco and Sun sales force: high-end servers with terabytes of storage all connected together in a network that has packets flying so fast you can feel your fillings melt if you get too close to the wires. The Archive, on the other hand, is a poster child for function over flash. The machines on the racks are the cheapest possible—the ideal computer is one with a reasonable CPU, a gigabyte of RAM and case for under $300—loaded with a free operating system. The racks look to be straight out of Costco and you won't find an Aeron chair anywhere on the premises.
Gathering this many computers is certain to point out even the smallest problems inherent in the hardware. There is a failure of one kind or another almost very other day. Hard drives are a common problem, failing long before the manufactures' specification claims they should. The summer-time temperatures in the Archive facility hover slightly above what would be covered by the warranty, and a failed drive is potentially a piece of internet history lost. Motherboards, power supplies: anything that can go wrong, will. In those cases, a crash cart is rolled over, the problem diagnosed and hopefully fixed.
Dim lightsEmbed Embed this video on your site The Internet Archive is a complete snapshot of all web pages on every website since 1996 till today. Since the average lifetime of a page on the Internet is 100 days, this snapshot is retaken every two months. The Internet Archive at BA includes the web collection of 1996 to 2006. It represents 1.5 petabytes of data stored on 880 computers. The entire collection is available for free access to researchers, historians, scholars, and the general public.
So why isn't the Archive using the latest and greatest Sun servers in a temperature controlled co-location facility where the air and power are filtered? The answer is simple: Librarians don't drive Porches. The Archive budget is not overly generous, and the total cost of ownership for Sun servers, proprietary operating systems and top-of-the-line routers and switches can be onerous. Filtering and cooling the air would approximately double the operating cost of the Archive. Under these restrictions it makes more sense to use commodity hardware and have layers of redundancy rather than a single server worth thousands of dollars.
Crawling
Anyone familiar with the concept of a web spider will understand the heart of the Archive's operation. Spiders are programs that traverse the internet and glean information from web sites. Search engines such as Google, Yahoo and Lycos rely on spiders to gather information to feed their search engines. The quality of these spiders and the quality of the indexing determine the success of your searches.
The Internet Archive does essentially the same thing in what it calls a "crawl." Crawls are carried out by the Archive itself or crawls are donated by other entities. Most of the data currently in the Archive has been donated by Alexa internet, also founded by Brewster Kahle. The Archive does two kinds of crawls: broad and narrow. A broad crawl is an attempt to archive a wide range of sites as completely as possible while a narrow crawl is designed for complete coverage of selected sites or selected topics. Both types of crawls have their own inherent challenges.
Dim lightsEmbed Embed this video on your site archive.org has many secrets beyond the Internet Way Back Machine, this episode gives you details on many of the other great services you can find, and all the free types of media for download!
Broad crawls can create a number of different problems. While it may be easy to create a crawler that takes full advantage of a 100Mbs link to the internet, it becomes increasingly difficult to keep the crawler fed. Extracting unique URLs becomes computationally demanding as the database of indexed sites increases. A broad crawl can encompass over 150 million web pages in a week and run for 40-60 days total. Each page encountered needs to have the links on it extracted and followed. However, those links first need to be checked against the database of previously visited sites. If it's the first time this URL has been hit, then it will need more attention while a repeat can be ignored. This comparison is done in RAM, so if the database is large enough to exhaust the RAM on the computer and force it to page to disk, the speed of the crawl slows down significantly.
Dim lightsEmbed Embed this video on your site At GreyThumb Silicon Valley in response to a question by Al Lundell, Bruce Damer puts forward my position that we currently exist in a post-singular world. Many thanks to Al Lundell for the footage and also the additional graphic overlays.
Another issue is for crawls is politeness—not all web sites are able to handle the load imposed by a high-performance, multi-threaded crawler. In these cases, there are only two outcomes: either the crawler is smart enough to back off and reduce the strain on the server, or the server will likely crash. A crawl that is looking up tens of millions of web sites per day can also having a devastating effect on DNS servers. These servers also often run out of RAM quickly as the lookups accumulate. Broad crawls are a necessity because in the vast pool of web pages, it's impossible to know what information should be preserved. A broad crawl is specifically designed to copy as much information as possible over a wide range of web sites. Since there are a lot of duplicate pages out there, and the task of discerning them on the fly is far to difficult, the Archive ends up with about a 30% duplication rate on broad crawls.
Narrow crawls may require less storage and less bandwidth, but they have their own challenges. On a topic-driven crawl, the most obvious is the programming involved in assuring that the crawl has achieved its goal. Making certain that topics such as the attacks on September 11th have been covered completely goes far beyond finding pages with those keywords on them.
Dim lightsEmbed Embed this video on your site Sun Project Blackbox The Internet in a Box Sun Microsystems IBM Cloud Computing Sun and the Internet Archive unveiled the Black Box "Wayback Machine". A digital time capsule, "The Wayback Machine" has been collecting "snapshots" of the web since 1996, allowing visitors to freely access archived versions of Web pages across time. One of the fastest growing digital libraries in the world, the Internet Archive migrated it's digital archive efforts onto Sun's open hardware and software platforms, and established a new primary datacenter that will be housed at Sun's Santa Clara, California campus. Bringing Open Storage and Cloud Computing innovation to the Archive, Sun helped the Internet Archive solve two important technological challenges: storing massive amounts of data and ensuring this data will be preserved in the future. Available video includes views of Sun's Modulated Datacenter undergoing testing in CA, as well as edited packages on this procedure and Project Blackbox.
Sure it's cool, but what can it do?
The Internet Archive is for web developers what home movies must be for celebrities. Preserved for posterity are our bad web designs, animated gifs and blinking text. The Wayback Machine is a glimpse into the pages held in Mission Street: enter a URL and it will sweep you back as far as 1996 to look at the early days of the internet explosion. The Wayback Machine is a popular site, it receives about five million hits per day. While it is clearly in heavy use, it's not known what research, if any, the people using the Wayback Machine are doing. Other researchers are using the Archive, but it's not an easy task. While there is plenty of data to look at, there isn't an easy interface for accessing it.
The types of research that are waiting to be done are even more interesting. The Archive contains over ten thousand news sites, various archives of e-mail lists, and a growing number of blogs. This is the chatter of the world, and as time goes on it can provide a wonderful glimpse into the psyche of the time. Nefarious uses are equally easy imagine: the effects of advertising campaigns, product releases or political debates. Having the opinions of millions of people clearly documented, easily accessible and quantifiable would be a boon to market researchers and anthropologists alike. Blogs and personal web sites may be of even greater interest than mainstream news site since they are an unfiltered view of the public at large.
