The Invisible Web
also referred to as
The Hidden or Deep Web
by Rebecca Landau, MLS
The term "invisible" refers to information on the web that search engines cannot or will not search.
- Information in a database
- Dynamically generated pages (Responses to a specific query)
- Dictionary definitions
- Cataloged library holdings
- Late breaking news
- Sites requiring a password, login, or fee
- Pages with a question mark, "?" in the URL
- Certain file formats (e.g., pdf, power point, word perfect, etc.)
Web Search Engines
Google and Alta Vista have spiders that search static pages. It is estimated that web search engines only search 40 to 50 % of the web. [Anne Clyde]
Web search engines:
- Do not find dynamic pages
- Respond to a specific query
- Do not search all file formats
General Periodical Site
- Magportal.com: http://www.magportal.com/
- Officially launched March 28, 2000, MagPortal.com is a search engine and directory for finding online magazine articles.
- Magportal.com has a help page http://www.magportal.com/help/ that explains the sort functions for retrieving articles (quality, date, publication, category).
Directories and Search Engines
Directories can lead to content that is hidden in a database. Use term "
database or databases" when searching a topic in a directory (e.g., Yahoo) or a search engine (e.g., Google) Example:
plane crash database
- Librarians' Index to the Internet: http://lii.org/
- Librarians' Index to the Internet is a directory. Example: literature and databases
"library catalog" and databases.
- Single Topic Website FindLaw: http://www.findlaw.com/
- FindLaw offers information on federal and state laws, bankruptcy, lawsuits and traffic tickets, lawyers and law firms.
- Example of a Search in FindLaw:
- Clicked on Legal Subjects. Clicked the Subject Heading
"Torts and Injuries" Clicked on
Databases. Led to a list of Databases. Clicked on
CHID Online Database Base and connected with a search screen.
- ERIC: http://www.eric.ed.gov/
- This is a free bibliographic database relating to education.The database has more than 1.1 million citations going back to 1966 and includes more than 107,000 full-text non-journal documents (issued 1993-2004).
References on the Invisible Web
- Robert J. Lackie, Those Dark Hiding Places: The "Invisible Web" Revealed,http://empconcepts.com/didl/Internet_module/ThoseDarkHidingPlacesTheInvisibleWebRevealed.pdf
- Invisible Web: What it is, Why it exists, How to find it, and Its inherent ambiguity, http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html
Books and Articles
- The Invisible Web: How to find the hidden resources that most search tools skip over, by Walter Minkel, School Library Journal, December 2002. p. 29
- Gary Price and Chris Sherman. The Invisible Web: Uncovering Information Sources Search Engines Can't See. CyberAge Books, July 2001. ISBN: 091096551X (Paper $29.95).
- The Invisible Web by Anne Clyde, Teacher Librarian, April 2002, Vol. 29 number 4.
Updated on July 31, 2015 by