The Internet provides access to a wealth of information on countless topics contributed by people throughout the world. On the Internet, a user has access to a wide variety of services: electronic mail, file transfer, vast information resources, interest group membership, interactive collaboration, multimedia displays, and more. The Internet consists primarily of a variety of access protocols. These include e-mail, FTP, HTTP, Telnet, and Usenet news. Many of these protocols feature programs that allow users to search for and retrieve material made available by the protocol.

For background information on Internet access protocols, see A Basic Guide to the Internet.

The Internet is not a library in which all its available items are identified and can be retrieved by a single catalog. In fact, no one knows how many individual files reside on the Internet. The number certainly runs into the many millions and is growing at a rapid pace.

The Internet is a self-publishing medium. This means that anyone with a small amount of technical skill and access to a host computer can publish on the Internet. It is important to remember this when you locate sites in the course of your research. Internet sites change over time according to the commitment and inclination of the creator. Some sites demonstrate an expert's knowledge, while others are amateur efforts. Some may be updated daily, while others may be outdated. As with any information resource, it is important to evaluate what you find on the Internet. For more information, see Evaluating Internet Resources.

Also be aware that the addresses of Internet sites frequently change. Web sites can disappear altogether. Do not expect stability on the Internet.

One of the most efficient ways of conducting research on the Internet is to use the World Wide Web. Since the Web includes most Internet protocols, it offers access to a great deal of what is available on the Internet.

HOW TO FIND INFORMATION ON THE INTERNET

1. Join an e-mail discussion group or Usenet newsgroup
2. Go directly to a site if you have the address
3. Browse
4. Explore a subject directory
5. Conduct a search using a Web search engine

1. JOIN AN E-MAIL DISCUSSION GROUP OR USENET NEWSGROUP

E-mail discussion groups tend to be associated with academic institutions. Many topics are scholarly in nature, and it is not unusual for experts in the field to be among the participants. In contrast, Usenet newsgroups cover a far wider variety of topics and participants have a range of expertise. Be careful to evaluate the knowledge and opinions offered in any discussion forum. Note also that a small number of e-mail groups are cross-posted as Usenet newsgroups. For example, the early music e-mail group EARLYM-L also exists as the newsgroup rec.music.early.

E-mail discussion groups are managed by software programs. There are three in common use: Listserv, Majordomo, and Listproc. The commands for using these programs are similar. For information on how to use listserver software, see the tutorial Internet from the VAX Prompt.

A list of Usenet newsgroups can be accessed from within a newsreader program. Using the RN reader on Unix, for example, you can type dir/group/all and receive a list of every newsgroup to which the University subscribes. For more information on the RN newsreader, see Using the RN Newsreader from the VAX.

A good Web-based directory to assist in locating e-mail discussion groups and Usenet newsgroups is Liszt, located at http://www.liszt.com/.

2. GO DIRECTLY TO A SITE IF YOU HAVE THE ADDRESS

This is the format of the URL: &nsp;   protocol://host/path/filename

For example:

http://cedr.lbl.gov/cdrom/doc/cdrom.html      a hypertext file on the Web
ftp://bongo.cc.utexas.edu/microlib      a file at an FTP site
telnet://library.albany.edu     a Telnet connection

3. BROWSE

4. EXPLORE A SUBJECT DIRECTORY

Subject directories differ significantly in selectivity. For example, the famous Yahoo! site does not consider content when adding Web pages to its database. In contrast, the Argus Clearinghouse collects and rates subject guides often compiled by experts. Consider the policies of any directory that you visit. One challenge to this is the fact that not all directory services are willing to disclose either their policies or the names and qualifications of site reviewers. A number of subject directories consist of links accompanied by annotations that describe or evaluate site content. A well-written annotation from a known reviewer is more useful than just a list of links.

Among the more prominent and useful directories are these: 

Recommended starting points:

• The Argus Clearinghouse is one of the highest quality subject directories on the Internet. This site consists of rated collections of recommended sites organized into subject-specific guides. The guide authors are often specialists in the field. This site is highly recommended for academic research.
• The WWW Virtual Library is one of the oldest and most respected subject directories on the Web. This directory consists of individual subject collections, many of which are maintained at universities throughout the world.
• INFOMINE is a large directory of Web sites of scholarly interest compiled at the University of California, Riverside. The directory may be browsed or searched by subject, keyword, or title. Each site listed is accompanied by a description.
• If you want to explore a large number and variety of sources, try Yahoo!. This is the largest subject collection on the Internet. Yahoo! indexes hundreds of thousands of links and organizes them in a hierarchical subject directory. Be aware, however, that Yahoo! accepts most sites submitted to it and do not check for quality or authority. In addition, Yahoo! does not attempt to provide a comprehensive listing on any topic. Its coverage of academic subject areas is generally sporadic. Certainly the Yahoo! database includes links to excellent resources. Each resource found here, however, must be evaluated carefully.

5. CONDUCT A SEARCH USING A WEB SEARCH ENGINE

Search engines located on the World Wide Web have become quite popular as the Web itself has become the Interneotj|t environment of choice. Web search engines have the advantage of offering access to a vast range of information resources located on the Internet. Many search engines compile a database spanning multiple Internet protocols, including HTTP, FTP, and Usenet. Web search engines tend to be developed by private companies, though most of them are available free of charge.

A Web search engine service consists of three components:

• Spider: Program that traverses the Web from link to link, identifying and reading pages
• Index: Database containing a copy of each Web page gathered by the spider
• Search engine: Software that enables users to query the index and that usually returns results in relevancy ranked order

With most search engines, you fill out a form with your search terms and then ask that the search proceed. The engine searches its index and generates a page with links to those resources containing some or all of your terms. These resources are usually presented in relevancy ranked order. A new development in search engine technology is the ordering of search results by concept, keyword, or site.

All search engines have rules for formulating queries. It is imperative that you read the help files at the site before proceeding. Online tutorials can also help you learn the rules. A short list of recommended tutorials appears at the end of this file.

Among the more prominent and useful search engines are these: 

1. Start with Infoseek. This is a very quick and accurate search engine that offers several field searching options and simplified keyword and phrase searching. Infoseek clusters all your results from one site into one result, so that each result comes from a different site. This makes it easy to scan for a variety of documents brought in by your search.
2. AltaVista is another good choice. This engine has a very large database, handles sophisticated Boolean searches, and offers several field searching options.
3. Also try Northern Light. This unique engine clusters results into Custom Search Folders, which contain specific subtopics and sites retrieved by your search. Northern Light therefore gives you quick access to aspects of your topic that interest you. Excite and HotBot are also recommended services to try.
4. MetaCrawler is a good site to try if your topic is obscure or if you want to retrieve results from a variety of search engines with a single search statement. This service searches multiple search engines simultaneously and offers useful search options. MetaCrawler returns your results in a single list and removes the duplicate files. This type of search processing is called multi-threaded searching. Other recommended multi-threaded search engines include Inference Find, MetaFind and ProFusion.
5. Try The Internet Sleuth if you want to search a topic-oriented database. This site offers access to hundreds of searchable databases in several subject areas.

HOW TO FORMULATE QUERIES

1. Identify your concepts

When conducting any database search, you need to break down your topic into its component concepts. For example, if you want to find information on the budget negotiations between President Clinton and the Republicans, these are your concepts: CLINTON, REPUBLICANS, BUDGET.

2. List keywords for each concept

Once you have identified your concepts, you need to list keywords which describe each concept. Some concepts may have only one keyword, while others may have many.

For example:

     CLINTON        REPUBLICANS         BUDGET
                    HOUSE SPEAKER       BUDGET NEGOTIATIONS 
                                        BUDGET BATTLE
                                        BUDGET IMPASSE
                                        BUDGET DEAL

Depending on the focus of your search, there may be other keywords you would wish to use.
3. Specify the logical relationships among your keywords

Once you know the keywords you want to search, you need to establish the logical relationships among them. The formal name for this is Boolean logic. Boolean logic allows you to specify the relationships among search terms by using any of three logical operators: AND, OR, NOT.

Search Statement              Result of search

World War I   AND             Files containing both these terms
World War II   

World War I   OR              Files containing at least one of these terms
World War II

World War I   NOT             Files containing the term World War I but
World War II                  not also the term World War II

Certain search engines allow you to use a proximity operator. This a type of AND logic which specifies the distance between words in a source file. For example, AltaVista and Lycos let you use the NEAR operator. Consider this search: Clinton NEAR budget. In AltaVista, the two terms must be within 10 words of each other in the source file. Lycos allows user-specified distances. Use of this option can help you gain relevance in your search results.

Most Web search engines cannot handle a single search statement that includes all the terms listed in Step 2 above. You may need to repeat your search a few times using terms in different combinations until you get results that are satisfactory. For example, you may start with CLINTON, REPUBLICANS, BUDGET NEGOTIATIONS and connect these terms with AND logic. Take a look at your results. If you are not finding what you want, repeat the search with alternative keywords for the budget concept. Your initial results may give you ideas about which new terms to try.

For more information on formulating searches, see Boolean Searching on the Internet.

TIPS ON CONDUCTING SEARCHES

1. Read the directions at each search site. The technique for formulating a search depends on the search engine you are using. There is a wide variety of options available among the different search engines.
2. If you have a multi-term search, be sure to determine which type of Boolean logic you should use. For example, a search about the relationship between latitude and temperature would be formulated as:    +latitude   + temperature on many Web search engines in order for AND logic to apply.
3. Include synonyms or alternate spellings in your search statements and connect these terms with OR logic.
4. Check your spelling.
5. Take advantage of capitalization if the search engine is case sensitive.
6. If your results are not satisfactory, repeat the search using alternative terms.
7. If you have too many results, or results that are not relevant:
• Add concept words
• Use vocabulary that is specific to your topic
• Link appropriate terms with the Boolean AND ( + ) so that each term is required to appear in the record
• Choose an option requiring exact term matches
• Use term proximity operators if they are available
• Narrow your search to individual parts of the Web page such as title, first page level, etc.
• Use the Boolean NOT to keep out records containing terms you don't want
8. If you have too few results:
• Drop off the least important concept(s) to broaden your subject
• Use more general vocabulary
• Add alternate terms or spellings for individual concepts and connect with the Boolean OR
• Choose an option allowing for loose or concept matches
9. Try different sources within search engines to diversify your results. Sources can include Usenet newsgroups, Internet FAQs, reviewed pages, and more.
10. Experiment with different search engines. No two search engines work from the same database.
11. You may want to try Web sites which allow you to search multiple search engines simultaneously. Be aware that you will lose access to advanced query options since not all engines offer them.

FOR MORE INFORMATION

The Finer Points of Web Search Engines

http://www.albany.edu/library/internet/finer.html

Finding Information on the Internet: A Tutorial

http://www.lib.berkeley.edu/TeachingLab/Guides/Internet/FindInfo.html

How to Choose a Search Engine or Research Database

http://www.albany.edu/library/internet/choose.html

Searching the Internet: Recommended Sites and Search Techniques

http://www.albany.edu/library/internet/search.html