FTS - File Tracking System

Version 5.31s, 31st July 2003

Since 2002 there has been an urgent need to port the application to a 32 bit language. To date, my choices were very limited due to cost and perceived use by others as there has been little feedback to an apparent complicated indexing method. However I have started to review the 32 bit options as by now FTS is starting to fail on some fast machines. This appears to be a chip architecture problem.

Project

This project began in 1998 but has its roots as far back as 1980. There was a need to store many scanned images, computer files plus indexed references to thousands of negatives and slides. The problem is not indexing or cross referencing but the ability to use the data/image/photo once you've found where it is. This system provides a link straight to file with requiring to remember the filename, directory or CD. Multiple CD searches can be a pain. Items are found either from the assigned page indexed to or a keyword search.

The key to the program lies in the storage and indexing strategy: Physical storage and logical indexing are split into two separate functions. Files and locations are tracked by the database. These entries are then indexed by logical method irrespective of file location.

Currently the programme is logging 74,000 files across 38 x CDs; all assembled since 1998. The program is designed to index 50,000+ computer files very quickly. This is beyond the scale of most people until collections become overwhelming. By then, it's too late.

I wrote the program to assist my hobby goals and decided that others may need a similar tool. So I'm offering it to those that may be interested. I'm only a programmer in the sense of creating programs to tackle my hobby logistics.

Indexing Strategy

The program was designed around the basic indexing philosophy of:

• How to find the stuff long after the user has forgotten the data entry
• So simple that a single pass of the index data entry will be sufficient
• On the assumption that 98% of typing is correct, mistakes are picked up at report time
• 'Drill down' indexing ensures rapid placement of items. As required refinements can be made depending upon use and other items that fit similar criteria

'Drill down' indexing is a reverse approach to indexing than that normally used. With a photo there is an attempt to accurately index the photo by what it shows or what the item is. So the focus is extremely close. By the method used with FTS, the subject is categorised into an extremely broad "GROUP", refined into a subgroup called a "TOPIC", then given a description.

The concept of 'drill down' indexing is where lots of items with hazy descriptions can be thrown together and only when sufficient items warrant special attention, can the user then deal with then by specifying another topic. The concept runs something like BATHTUB -> BUCKET -> CUP -> THIMBLE. For example all cars, trucks road machinery can be placed into a group called 'ROAD'. Subgroups would then be cars, trucks, cranes, forklifts, construction, maintenance, signs, views etc. The descriptions are then used for keywords and other interesting information. There are extra fields for date, location, etc which assist with prompting for information about the subject.

Material specified this way then uses ALL the information as a description. This allows the item description to have few words, which can also be used as keywords in a search. The combination of all the fields available allow the user to specify broad or narrow searches as required. The use of groups and topics allow the searches to be fully described.

Program Features Summary

Program positives:

• File access is not by direct use of the filename but from a HTML page link File associations open the approriate application.
• This is an archive storage method that allows users to log files and material. For projects and special efforts, the FTS method allows quick searches and/or retrieval of relevant material
• Look up tables via mouse during edits saves time and reference lists. Useful for low use references. It's still quicker to type the reference codes.
• Checks a directory and adds files into the database for indexing
• Allows the user to assemble files to fill and write a CD; from manual ID entry, keyword search or both.
• The program tracks the files on HDD and when the CD is written, the changes are reflected in the reports.
• Allocates a unique ID to each image and text based index reference.
• Generates HTML 'reports' that link the each entry to a file where ever it is. The 'no surprises' output shows the disk the file is on with the link. By including the disk data in the link, a failed link will show enough information to jog the memory.
• A database search creates a temporary database which can be saved and used to generate export material. This database is used to transfer files back from CD to HDD and is also used to create HTML pages to link the files for offsite CD use.
• If file swapping is done with people using the same program, images can be transferred and the indexed images can be appended to the bottom of the database where the material was received.
• Easy cross referencing of images. Allows multiple references to the same file for different subjects within a photo/scan/image
• Allows negatives to be indexed before scanning. As required, the photo can be scanned and the same negative entry changes from text to a link
• Capacity of 1,000,000+ files. Test program ran to 1.2 million. More records with cross referencing.
• Database searches return the approximate size of files ( in Kb, Mb or Gb ) if a download is required.
• Transfers from CD or removable media to HDD are conducted by the program. The program prompts for the disk number as files are downloaded.
• Indexing evolves to suit the user and can cover any subject
• For some actions a Logfile runs in the background saving changes made during edits.
• Images or files retrieved within 3-5 mouse clicks ( plus scrolling ), usually access within 20 seconds.
• Automatic program backups

  Program negatives:

• Program seems to have problems on fast machines or with Windows2000. Not experienced personally and only one report to date. This was a Pentium 2GigaHz, AMD chips seem OK
• Standalone 'EXE' program works in Microsoft Windows(tm) 3.1/95/98/Me
• Looks a bit 'clunky'
• Mix of keyboard and mouse actions
• Reversed HTML page access ( links start on the right hand side:  reduces mouse movements by 15-20% )
• Minimum fields. Main description field is 65 characters. Program is used as an index TO data and files, not as a repository of data itself.
• Familiarisation - This program can be run in parallel with any other system being used. FTS probably requires about 4-6 weeks of familairisation. Not a lot really when considered the time saving for use and searches over 5+ years.

Samples

Screen Shot of Data Entry

Here's a screenshot of the data entry page. Other screen shots are contained in the manual, within the zipfile

• Page and screen layout

  This example is the Group 'Rail - Victoria' linking to topics of the group. The selected topic 'Locomotives - Diesel' links to the Item Page which appears on the lower left frame. Links show access to file with DiskID far left of item. Text items show unscanned material. Note: this screen shot has been edited and does not show all scroll bars.

• Topic listing

  Edited list of Topics showing headings and codes used. The headings and three character codes are left to the user to organise.

• Statistics report

  Generated as required to view 'what's happening'.

Development

The latest developments are:

• Topic Global does not overwrite topics now
• Series add was uncompleted. Unfinished code found and work done. Now OK.
• Edit Return - the return button on the data window allows the user to return to the last record edited and continue on. Program sets up the index order then goes to the last record.
• Single audit menu choice - all six audits have been consolidated for ease of use. Single auditing still possible.
• Backups now progress from 1 through to 999 then repeat. The maximum backup value is used to delete the oldest <nth> backup to maintain a continuous file date order for backups
• Segregation of files into removable media blocks, where the storage capacity of the media is stored by the program. ie when files exceed ONE disk ( floppy, CD, DVD, etc ) the program allocates files to each disk and prepares an export file for each disk. This allows a large number of files to be transferred BUT treats each disk as an individual unit for the purposes of FTS 'Import' if used by another FTS program.
• Automatic consolidate within a Topic or Series using a first word match to an existing link or new page creation. Can also use a nominated amount of left hand characters to group all similar items. This Series page grouping reduces report times by placing similar items onto a page of their own thus reducing Topic page sizes.
• Ability to select and use CD's that have the same directory structures. Until now, directories had to be unique to reduce mixed disk problems (ie, you THOUGHT you had 'Disk 10' when really 'Disk 11' was in place ). Now it's possible to handle 'legacy' disks.

There will always be a few bugs. These are generally being weeded out as time goes. Let me know about them and I'll correct. I have no issue with fault finders because I test for some actions but neglect others that are rarely used. In case of trouble quit and start again.

Download

Note: for use on Microsoft Windows(tm) environment only.

There are two files available for download:

1. Download fts_app.zip - [100Kb]

   This file contains the empty datafiles and sample HTML files. Download this file if you have never run FTS before. It contains the empty datafiles and sample HTML files. Caution - if FTS is already running these files are NOT required again.

2. Download fts_exe.zip - [approx 1.5Mb]

   This file contains the program files and the manual, and will be the latest version to date.

   Download this file to run FTS. If you already have FTS running unzip this file into the working directory. The old FTS files will be overwritten and when the program is started it will check all the file structures and modify them as required. Fts_app.zip and Fts_exe.zip must be unzipped into the same directory.

   Note: It would be advisable to do a directory backup before unzipping files. If there are problems, the old directory and files can be re-used. FTS backups are for data and files only. Old FTS backups may not work with newer program versions due to field changes.

    

Upgrade

To upgrade an older version of FTS, download the 'fts_exe.zip' file and unzip it into the directory FTS is running from.

Freeware

Permission is granted to use the software and distribute the zipfiles. Copyright remains with the author.

pjv101@netspace.net.au

Peter J. Vincent, Melbourne, Australia