Just read this article and am intrigued.
Here are some quotes:
“The darkweb”; “the deep web”; beneath “the surface web” – the metaphors alone make the internet feel suddenly more unfathomable and mysterious. Other terms circulate among those in the know: “darknet”, “invisible web”, “dark address space”, “murky address space”, “dirty address space”. Not all these phrases mean the same thing. While a “darknet” is an online network such as Freenet that is concealed from non-users, with all the potential for transgressive behaviour that implies, much of “the deep web”, spooky as it sounds, consists of unremarkable consumer and research data that is beyond the reach of search engines. “Dark address space” often refers to internet addresses that, for purely technical reasons, have simply stopped working.
Michael K Bergman, an American academic and entrepreneur, is one of the foremost authorities on this other internet. In the late 90s he undertook research to try to gauge its scale. “I remember saying to my staff, ‘It’s probably two or three times bigger than the regular web,”‘ he remembers. “But the vastness of the deep web . . . completely took my breath away. We kept turning over rocks and discovering things.
In 2001 he published a paper on the deep web that is still regularly cited today. “The deep web is currently 400 to 550 times larger than the commonly defined world wide web,” he wrote. “The deep web is the fastest growing category of new information on the internet … The value of deep web content is immeasurable … internet searches are searching only 0.03% … of the [total web] pages available.”
Meanwhile the search engine companies are restlessly looking for paths into the deep web and the other sections of the internet currently denied to them. “There’s a deep implication for privacy,” says Anand Rajaraman of Kosmix. “Tonnes and tonnes of stuff out there on the deep web has what I call security through obscurity. But security through obscurity is actually a false security. You [the average internet user] can’t find something, but the bad guys can find it if they try hard enough.”
Ten years ago Tim Berners-Lee, the British computer scientist credited with inventing the web, wrote: “I have a dream for the web in which computers become capable of analysing all the data on the web – the content, links, and transactions between people … A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.” Yet this “semantic web” remains the stuff of knotty computer science papers rather than a reality.
It seems likely that the internet will remain in its Gold Rush phase for some time yet. And in the crevices and corners of its slightly thrown-together structures, darknets and other private online environments will continue to flourish. They can be inspiring places to spend time in, full of dissidents and eccentrics and the internet’s original freewheeling spirit. But a darknet is not always somewhere for the squeamish.
I knew there was a lot to the web that I didn’t know about but I had no idea that it was so vast. Read the article, its very interesting but tends to paint a not so rosy picture of the ‘dark web’. I’m interested in checking this it out but maybe a bit squeamish.
Will my curiosity be happy not knowing or will I have to take a look?