  
	
  
  
	
	  Overview of Programs
	
	
	  ht://Dig Copyright © 1995-2004 
THANKS.htmlThe ht://Dig Group 	  Please see the file 
COPYINGCOPYING  for
	  license information.
	
	
	
	  There are several programs in the ht://Dig package. 
	
	
	  
htdig.htmlhtdig 	
	
	  Digging is the first step in creating a search database. This
	  system uses the word 
digging while other systems call
	  it 
harvesting or gathering. In the ht://Dig
	  system, the program 
htdig.htmlhtdig  performs
	  the information gathering stage. In this process, the program
	  will act as a regular web user, except that it will follow
	  
all hyperlinks that it comes across. (Actually, it
	  will not follow all of them, just those that are within the
	  domain it needs to gather information on...)
	   Each document it goes to is examined and all the unique
	  words in this document are extracted and stored.
	
	
	  The digging process will 
only follow links and has
	  no notion of JavaScript, applets, or user-input forms.
	
	
	
	  
htsearch.htmlhtsearch 	
	
	  Searching is where the users actually get to use all the
	  information that was gathered during the dig and merge
	  stages. The 
htsearch.html	  htsearch
 program performs the actual searches. It typically
	  produces 
HTML output which will be seen by the
	  users, though other text formats could be generated by
	  editing the output templates.
	
	
	
	  
htmerge.htmlhtmerge 	
	
	 Merging does exactly that--it merges one database
	 into another. In previous versions of ht://Dig, the htmerge
	 program also formed databases for use by htsearch from the
	 htdig output. This process is now largely unnecessary except
	 for removal of invalid URLs which is now done by the htpurge
	 program.
	
	
	
	  
htpurge.htmlhtpurge 	
	
	 Purging removes documents and the associated words from the
	 databases. This should be done after running htdig to remove
	 invalid URLs, documents marked not to be indexed, old
	 versions of modified documents, etc. You can also specify
	 specific URLs to be removed explicitly by htpurge.
	
	
	
	  
htload.htmlhtload 	
	
	  Loading involves importing the contents of the databases
	  from formatted ASCII text documents as created by htdump or
	  the -t flag from htdig. This is, of course, destructive by
	  nature and data from the text files will replace any
	  conflicting data in the databases.
	
	
	
	  
htdump.htmlhtdump 	
	
	  Dumping involves exporting the contents of the databases to
	  formatted ASCII text documents. This can be useful for
	  backups, transferring databases between different operating
	  systems, changing the compression or encodings in the
	  ht://Dig configuration, parsing by external utilities. It is
	  
not recommended to edit these files by hand, so be
	  warned! (Minor edits will probably be fine.)
	
	
	
	  
htstat.htmlhtstat 	
	
	  The htstat program returns statistics on the databases,
	  similar to the -s flags for some of the programs. In
	  addition, it can return a list of URLs in the databases.
	
	
	
	
htnotify.htmlhtnotify 	
	
	  The ht://Dig system includes a handy reminder service which
	  allows HTML authors to add some ht://Dig specific 
meta.htmlmeta
	  information
 in HTML documents. This meta information is
	  used to email authors after a specified date. Very useful
	  to maintain lists that contain those annoying "new"
	  graphics with new items. (Hint: Things really aren't all
	  that new anymore after 6 months!)
	
	
	
	
htfuzzy.htmlhtfuzzy 	
	
	  To allow the searches to use "fuzzy" algorithms to match
	  words, the 
htfuzzy.htmlhtfuzzy  program can
	  create indexes for several different algorithms.
	
	
	Last modified: $Date: 2004/05/28 13:15:17 $
  
