“ NASA is indexing the ‘Deep Web’ to show mankind what Google won’t” exclaims the news headline, no doubt leading to much confusion amongst it’s readers. In 2015, the US Defense Advanced Research Projects Agency (DARPA) announced a partnership with NASA for a ‘Memex’ search engine to take things even further, indexing scientific materials, semantic meaning of multimedia content and even previous versions of websites potentially hosted by to create ever more powerful search technology. In the intervening years, Google’s Page Rank and successor algorithms have been in a continuous state of improvement, with the specifics guarded as one of the tech giant’s most precious pieces of intellectual property in this commercially competitive space. Bergman would later go on to use the analogy of an iceberg to illustrate the how ‘deep’ this content was, as how the vast majority of information was ‘below the surface’. In ‘The Deep Web: Surfacing Hidden Value’ Bergman explains how the ‘deep web’ may be searched via methods such as passing queries into search forms to retrieve dynamic results, searching corporate intranets, accessing private content behind logins and pay walls and other types of sites such as classified ads, ecommerce, job boards and forums. He divided the web into two parts, the parts traditional search engines were indexing he named the ‘surface web’, whereas his new ideas were about exploring the ‘deep web’. Bergman was designing the next generation of internet search algorithm which would go beyond the scope of traditional web crawler technology. ‘Deep web’ in a search context was coined in the year 2000 by technologist and internet search researcher Michael Bergman. internet forums 2011–2017 History Deep web search However it turns out it’s more than just terminology that people are confused about. One common misconception about the dark web and the deep web is that these two terms are interchangeable. Lexicographer Jane Solomon reports in mid 2015 that ‘deep web’ now so often appears in place of or in combination with ‘dark web’ that it is taking on a life of its own. Today the terms are incredibly nebulous with technology companies, darknet sociologists, technology journalists and Wikipedians all attempting to tease apart the overlapping and confusing definitions. However, in its original sense, it referred to web content inaccessible via traditional search engines. onion websites build on top of the Tor dark web. The term ‘deep web’ most commonly refers to hidden service. You’ve probably already heard of the ‘deep web’, a hidden part of the internet filled with criminality, mystery, advanced encryption technologies, hackers, government secrets and maybe even more? But how mysterious it is really? Is it the same as the ‘dark web’? Isn’t search indexing supposed to be involved at some point or it the ‘deep web’ something else altogether? This is my world and the world of the ‘deep web’. I have experienced the confusion and ambiguity surrounding the subject on a daily basis, prompting my investigations into underground darknet markets, creatively horrific scams and seen the birth of new generation of conspiratorial technology-driven paranoia, bred by half-truths, misunderstandings and fabrications in the absence any kind of real authority on the matter. It is a compilation of materials previously written by me on this blog, elsewhere and RationalWiki.Īs a cybercrime researcher, and amateur techno-sociologist, I’ve run a popular forum on the topic of the ‘deep web’ for almost two years now. Alas only about a quarter of the content made the cut, thus I present the complete essay submission here. In May this year my writing was feature in my first academic-level publication, the ‘ The SAGE Encyclopedia of the Internet ’, providing a lengthy definition of term ‘deep web’. The Deep Web - etymological curiosity and urban legend
0 Comments
Leave a Reply. |