The Meta Project is so large
that many man-years will be required
for anything like a full-scale implementation.
On the other hand, if the basic ideas are sound,
it should be possible to implement a subset
which demonstrates their essential validity.
So, I have begun development on the Meta Demo
The demo is a specialized "search engine"
for information on operating system software.
Given a manual page name, file system path name,
or keyword specification,
the demo retrieves a set of related items
(e.g., path names and man page names),
complete with terse annotations.
At present, the demo covers only some base distributions
of FreeBSD and Mac OS X.
Coverage of the Ports Collection
and/or other operating systems are under active consideration.
I am also looking into the issue
of providing browsing capability into the files themselves.
Despite its severely limited charter,
the demo has proven to be both interesting and useful.
Even as an experienced Unix user,
I find that the demo brings up items
which I either did not know about
or might not have associated with the item in question.
Consequently, I have decided to promote the current version
of the demo as "The FreeBSD Browser"
If significant numbers of FreeBSD users decide
that the browser is useful,
I will have achieved a basic "proof of concept"
for Meta's basic notions.
Also, both the usage logs and the explicit feedback
that these users provide should be very valuable
in improving the database, rule sets, and code.
The demo is implemented as a Perl-based CGI script,
fed by a host of information-gathering filters.
is used to store
a tied hash of (line-oriented and XML format) metadata;
to parse the returned data.
The metadata comes from:
- annotation files
- hard-coded rules
- system information
Thus, a link might be displayed because of
an annotation file entry,
a rule in the CGI script,
or a "FILES" entry in a man page.
To minimize the pain of waiting for results,
the CGI script uses a simple form of caching;
the HTML for (most) displayed pages is saved for future use.
As of mid-February 2001,
the demo's code base contained (roughly):
- 2,500 lines of Perl
- 20,000 lines of hand-edited XML
- 1,500,000 lines of generated XML
Although I try to code cleanly and document adequately,
the demo's code base is not ready for public viewing.
Nor, given my limited resources and large task list,
is the code likely to be "released" any time soon.
Interested individuals are welcome to contact me, however,
if they wish to assist in the project.
Although the demo's long-term goals are subsumed
in those of the Meta Project,
some short-term goals are worth mentioning:
- better tuning of the graph traversal rules
- graphical display of selected "subsystems"
- increased coverage and detail in the metadata
If the demo attracts enough users,
the current implementation could become strained.
Fortunately, several steps can be taken
to improve its performance:
- switch to faster (and perhaps more) hardware
- supply the server(s) with better bandwidth
- force production and caching of popular pages
- recode to use FastCGI, an RDBMS, etc.
In the medium term,
I hope to greatly expand the demo's depth and breadth.
Online browsing of cross-linked files
(e.g., source code and documentation) is quite feasible.
Coverage of add-on packages and other operating systems
(e.g., BSD, Linux, or even proprietary Unix variants)
is also a strong possibility.
-- Main.RichMorin - 16 Jun 2003