CGI Interface Documentation
Don't have an account yet? Click here to register. If you already have an account but you've lost your password, click here to reset it.
| News: |
| (no news) |
 | « Back to main |  | « Helpful Documents |  | « CGI Interface » |  |
CGI Interface Documentation
Table of Contents
- Interactive interface.
- Get all of a document ("raw form") using the Lemur internal id.
- Get all of a document ("raw form") using the Lemur external id.
- Get corpus statistics for a specified term.
- Specify which index to use.
- Print results without HTML tags (easier to parse).
- Get corpus statistics for two terms.
- Get a specified inverted list from a specified index.
- Print the CGI program's help message.
- Get a specified title inverted list.
- Get a specified title inverted list with positions.
- Get a specified anchortext inverted list from a specified index.
- Get a specified anchortext inverted list with word
- Batch several commands in a single request
Introduction
The CGI program supports an extremely simple interactive interface, as well as a simple API. The interactive interface is accessed via a http URL such as: http://education.lemurproject.org/search/sample/lemur.cgi . The underlying API commands are accessed by appending a '?' and a set of parameters to the CGI URL. The parameters specify operations to perform on a search engine index, for example, to lookup an inverted list.
Results are returned in a Web page that contains a header (everything before the <BODY> tag), the results, and a footer (everything after the <HR> tag). The header and footer can be ignored. They are the same for every command, and their only purpose is to identify the software.
The CGI interface supports the following functions:
The interactive interface supports query-based search.
http://education.lemurproject.org/search/sample/lemur.cgi
Get the original form of a document using the Lemur internal id (e.g., document 123).
http://education.lemurproject.org/search/sample/lemur.cgi?i=123
The results portion of the Web page is the original text of the document, including all markup.Get the original form of a document using the Lemur external id (e.g., document GX066-76-3331850).
http://education.lemurproject.org/search/sample/lemur.cgi?e=GX066-76-3331850
The results portion of the Web page is the original text of the document, including all markup.Get the corpus statistics for a term, for example, the term `star'.
http://education.lemurproject.org/search/sample/lemur.cgi?c=star
The results portion of the Web page contains the corpus statistics for the term.Specify which index another command should use by prepending a corpus qualifier. For example, to get the corpus statistics for a term, for example, the term `star', from index 3 (using a zero-based index, so that the first index is index 0), use the following command:
http://education.lemurproject.org/search/sample/lemur.cgi?d=3&c=star
The results portion of the Web page contains the corpus statistics for the term, taken from index 3.Print the results without HTML tags (easier to parse) by prepending the command g=p to any other command. g=p shifts graphics output ("g") into a format easier for programs ("p") to parse.
http://education.lemurproject.org/search/sample/lemur.cgi?g=p&c=star
The example above displays the corpus statistics for 'star' from the first (default) index.Print the CGI program's help message
http://education.lemurproject.org/search/sample/lemur.cgi?h
Get the corpus statistics for two terms with one call, for example, the terms `star' and 'trek'.
http://education.lemurproject.org/search/sample/lemur.cgi?c=star&c=trek
The results portion of the Web page contains the corpus statistics for the two terms.Your program may prefer to see the output in "program" mode.
http://education.lemurproject.org/search/sample/lemur.cgi?g=p&c=star&c=trek
The results portion of the Web page contains the corpus statistics for the two terms, in the order specified in the parameter list.Get the inverted list for a query term (e.g., `star') from a particular index (e.g., index id 1 - the second index). If no index is specified, index 0 is assumed.
http://education.lemurproject.org/search/sample/lemur.cgi?d=1&v=star
The results portion of the Web page contains the inverted list. An example inverted list is shown below.
The term `star' occurs 2047 times in the corpus. It occurs in 340 documents. It occurs in document 7 only 1 time within the main body of the document, while in document 22, it occurs 10 times within the main body.ctf df 2047 340 docid # mainbody mainbody_freq tf (mainbody) 7 1 1 1 9 1 1 3 17 1 1 1 22 1 1 10
Get the inverted list for a query term (e.g., 'star') where it occurs in the title field.
http://education.lemurproject.org/search/sample/lemur.cgi?v=star.title
The results portion of the Web page contains the anchortext inverted list for the word. An example inverted list is shown below.ctf df
The word 'star' occurs in the title field for document 599 one time.
17156 11041
docid # title title_freq tf (title)
599 1 1 1
2692 1 1 1
: : : :
5392 1 1 1Get the inverted list for a query term (e.g., 'star'), with positions, where it occurs in the title field.
http://education.lemurproject.org/search/sample/lemur.cgi?V=star.title
The results portion of the document contains the title's inverted list for the word. The format is similar to the v= argument, except that the tf and tf (title) fields are followed by a list of word locations indicating where the term occurred.
The word positions are with respect to the field within the document they are gathered from, so they are only partially useful. Their main value is that they support proximity matching, e.g., one could find a title that matches `#1(white house)'.Get the anchortext inverted list for a query term (e.g., `star') from a particular index (e.g., index 1). If no index is specified, index 0 is assumed.
http://education.lemurproject.org/search/sample/lemur.cgi?d=1&v=star.inlink
The results portion of the Web page contains the anchortext inverted list for the word. An example inverted list is shown below.
Document 2209 is 2682 words long. The word `star' occurs in it 14 times. There is 1 hyperlink to document 2209 in the corpus. The hyperlink anchortext contains the word `star' one time. Document 26584 is 713 words long. The word `star' occurs in it 30 times. There are 251 hyperlinks to document 6584 in the corpus. The word `star' occurs 22 times in the anchortext for 22 hyperlinks.ctf df 116132 48797 docid doclen tf # inlinks inlink_freq tf (inlink) 4 305 1 2 0 0 79 235 1 1 0 0 : : : : : : 2150 252 1 0 0 0 2209 2682 14 1 1 1 : : : : : : 6584 713 30 251 22 22 : : : : : : Get the anchortext inverted list with positions for a query term (e.g., `star') from a particular index (e.g., index 1). If no index is specified, index 0 is assumed.
http://education.lemurproject.org/search/sample/lemur.cgi?d=1&V=star.inlink
The results portion of the Web page contains the anchortext inverted list for the word. The format is similar to the va= argument, except that the tf and tf (inlink) fields are followed by a list of word locations indicating where the term occurred.All of the anchortext from the various hyperlinks is concatenated together into a "virtual" anchortext document. The word positions are with respect to this virtual anchortext document, so they are only partially useful. Their main value is that they support proximity matching, e.g., one could find anchortext that matches `#1(white house)'.
In general, key/value pairs can be chained together and will be processed in left to right order. This batching was already supported for some commands in the past, but others, like the "I" command, did not work. This was recently fixed so that fetching documents with the "I" command can also be batched. Here are a few examples using the "I" command with different output formats.
The following will display documents 1, 2, and 3 - one document per line in plain text. (This is probably the cleanest format for retrieving multiple documents.)
http://education.lemurproject.org/search/sample/lemur.cgi?g=d&I=1&I=2&I=3
The following will display documents 1, 2, and 3 - but with each document separated by individual / tags (View the source of the output in your browser.)http://education.lemurproject.org/search/sample/lemur.cgi?g=p&I=1&I=2&I=3
A bit more fancy: the following will display document 1 from each of the four corpora (no stop, no stem; no stop, with stem; with stop, no stem; and with both stopwords and stemming).http://education.lemurproject.org/search/sample/lemur.cgi?g=d&d=0&I=1&d=1&I=1&d=2&I=1&d=3&I=1
© 2007 - The Lemur Project (http://www.lemurproject.org)