Posts Tagged 'Web Searching'

Mining the Web with Ruby

Mining the Web with Ruby

Wayne Graham and Joe Gilbert of the Scholars’ Lab continue their workshops on Ruby, a web programming language, with a look at tools for programmatically acquiring information from the web.
Tuesday, March 23 at 3:00 p.m.  in the Scholars’ Lab Classroom.

For more information contact the Scholars’ Lab at (434) 243-8800 or Send a Message.


Image released under a Creative Commons license by Flickr user simpologist.

Guidebook to Internet Searching

You can view or download in PDF format a 30+ page book by Dean Sherwin called Guidebook to Internet Searching.  This book outlines basic techniques for searching the internet for information of all sorts, including web pages, images, people, products, videos and data files.

Word Order Matters

The following is reposted from the ResearchBuzz blog for January 13, 2010 by Tara Calishain.


Doing Real Time Search? Watch Your Word Order

Posted: 13 Jan 2010 04:36 AM PST

If you’ve been reading ResearchBuzz for a while, you probably know that the way you enter your search terms in Google makes a difference.  If you enter words in one order, you may very well get a different result count and a different order to the results you get back.  (Try searching Google for scratching post and post scratching to get an idea of what I’m talking about.)

I have used this knowledge to benefit over the years, when I needed to narrow down search results or just get a different perspective on what was available.  When Google’s new real-time search came out, I assumed word order would no longer make a difference.  After all, real-time search is just that — the latest and greatest material that Google is adding to its index.  The stream should be the stream, right?  No matter what kind of word order you use.

Turns out that’s incorrect; Google does change the real time search results based on your word order.  That’s okay, but it does mean if you’re looking for real-time data you may want to play around with your word order, especially if you’re searching for words that don’t make a common phrase.

Let’s take an example.  I’m interested in a Ford Taurus, and I want to see what kind of real-time buy/sell activity there is out there.  I do a Google search for Ford Taurus and pay attention to the latest results.


I’m getting the “latest” results, and the list looks very much like a Google search result except the results show how recently the content was indexed.  The result count for this search, at this writing, is 4,250,000.  You’ll also notice that the left nav gives you related searches, mostly other car models.

Now take that search and turn it around.  Just turn it and do a search for Taurus Ford.  Your search results now look like this:


You’ll note that the related searches are gone, the search results have shot up to about 6,670,000 results, and the order of the search results has shifted a little bit.

Now, is this bad?  No, of course not.  But if you’re really working in the live search and you want to make sure you get as many search results as you can, you’re going to have to run multiple searches of multi-word queries.

Word order shows a lot of difference when the words make up a phrase.  If you do a search for search engine, at this writing you’ll get about 315,000,000 results along with some Twitter tweets.  If you change the search to engine search, the result count drops to 109,000,000, the results shift around a lot, and only one tweet appears, way down at the bottom of the page.

I remember being astonished when search engines hit a billion pages of indexed content, but that’s nothing these days.  The name of the game continues to be narrowing down your results to get the information you need and approaching a search problem from different angles.  You can make a different angle just from changing the word order in your query even in Google’s real-time search; try it!

Info Tool of the Week: Science Search Engines

Everybody knows about Google — and everybody uses Google to search the web for needed information.  But wouldn’t it be nice if, sometimes, you could search the web with a search engine optimized for just science and engineering materials?

Well — you can!  Here are some science and engineering oriented search engines that can help you search the web more efficiently and effectively.

Scirus – for scientific information only.  According to their site, Scirus is the most comprehensive scientific research tool on the web. With over 350 million scientific items indexed at last count, it allows researchers to search for not only journal content but also scientists’ homepages, courseware, pre-print server material, patents and institutional repository and website information.  You can also get the latest scientific news from the New Scientist magazine.  Use the Advanced Search option to set your preferences for type(s) of materials to search for, date ranges, etc.  Scirus is sponsored by the Elsevier publishing group.

TechXtra is a UK-based service which can help you find articles, books, the best websites, the latest industry news, job announcements, technical reports, technical data, full text eprints, the latest research, thesis & dissertations, teaching and learning resources and more, in engineering, mathematics and computing.  Many of the things you’ll find through TechXtra come from the ‘Hidden Web’, and are not indexed by Google.

Intute is another UK-based search engine sponsored by a consortium of British universities and libraries.  In addition to providing subject-based web searching Intute offers a variety of online training sessions, customized user options and alerting services, podcasts, news feeds and related services. is a web search engine maintained by the United States federal government designed specifically to search for U.S. government-produced web pages. is a gateway to government science information and research results. Currently in its fifth generation, provides a search of over 40 scientific databases and 200 million pages of science information with just one query, and is a gateway to 1,950+ scientific Websites.  The content for is contributed by participating agencies committed to serving the information needs of the science-attentive citizen, including science professionals, students and teachers, and the business community.

The above science and engineering search engines are only a few of the tools you can use to optimize your web searching and save yourself time and frustration.  For more information about these and other web search options, come by the Brown Science and Engineering Library and let us show you how to find the information you need for your research!

Dating Google

The following is taken from the The Internet Tourbus, Vol. 15, No. 57, September 17, 2009:

When you search for something in Google, you normally have no idea how old the results are.  The highest-ranked results may obsolete or not relevant to you, because of Google’s page-ranking criteria.  Wouldn’t it be cool if Google let you find documents based on their age, or search for Web pages that were created in a specific date range?

Thanks to a relatively new Google search feature, now you can search Google by date and time. Here’s the scoop on date-based searching…


RSS Feed

July 2020