Wedding Picture
September 13th, 2005
Marriage
September 13th, 2005This is a transcript of the vows from our wedding:
OFFICIANT:
We gather here today to celebrate the union of James and Gail as they publically proclaim their love. By this union, they begin a new part of life’s journey - a time wherein they look not with their own eyes, but with the eyes of the other. In this, they see the world not only as themselves, but together as one. By this bond, James and Gail hold each other in equality, with honor and compassion, forsaking all others, as they walk together through life.
All of you are James and Gail’s community, and each of you has played some part in bringing them to this moment. This is why gathering as a community is such an important part of this day. James and Gail are now taking a new form as a married couple, and in this form, they become part of their community in a new way.
James and Gail, we are here to remember and rejoice with you and to recount with one another that it is love that guides us on our path, and to celebrate as you begin this journey together. It is in this spirit that you have come here to today to exchange these vows.
VOWS
James, repeat after me:
I James take you Gail to be my wife.
I promise above all else to live in truth with you and to communicate fully and fearlessly.
I give you my hand and my heart as a sanctuary of warmth and peace and pledge my love and devotion in honor as I join my life to yours.
Gail, repeat after me:
I Gail take you James to be my husband.
I promise above all else to live in truth with you and to communicate fully and fearlessly.
I give you my hand and my heart as a sanctuary of warmth and peace and pledge my love and devotion in honor as I join my life to yours.
RINGS
For thousands of years lovers have exchanged rings as a token of their vows. These simple bands are not of great value in themselves, but are made precious by our wearing of them. Your rings say that even in your uniqueness you have chosen to be bound together. Let these rings also be a sign that love has substance as well as soul, a present as well as a past, and that, despite its occasional sorrows, love is a circle of happiness, wonder, and delight.
James, take Gail’s ring and put it on her finger, and repeat after me:
Just as this circle is without end, my love for you is eternal. Just as it is made of indestructible substance my commitment to you will never fail. With this ring I take you to be my trusted confidante and partner for life.
Gail, take James’s ring and put it on his finger, and repeat after me:
Just as this circle is without end, my love for you is eternal. Just as it is made of indestructible substance my commitment to you will never fail. With this ring I take you to be my trusted confidante and partner for life.
James, you may now kiss your bride.
(Signing of register)
PRONOUNCEMENT
James and Gail, although I’m officiating here today, it is not truly in my power to sanctify, legitimize or bless your relationship in any way, because the two of you have already done that in your hearts. So, by joining hands right now and looking into each other’s eyes, let it be known that you are joined - body, soul, and spirit - in this lifetime, and that this bond is sacred and eternal. And now that you have stood before me and exchanged these rings and these vows, and have agreed to be married according to the laws of Ontario, it gives me gives me great pleasure to announce you as Husband and Wife.
Brief Tutoral on Installing thttpd 2.25b
March 25th, 2005To install thttpd:
wget http://www.acme.com/software/thttpd/thttpd-2.25b.tar.gz groupadd www tar -zxf thttpd-2.25b.tar.gz cd thttpd-2.25b ./configure --prefix=/usr/local/thttpd --mandir=/usr/local/man make make install
Then you can start it with:
/usr/local/thttpd/sbin/thttpd -p 80 -d /usr/local/thttpd/www
The -p option is the port you want the server to run on, and the -d option is for the root directory you want it to serve pages from.
Edit /usr/local/thttpd/www/index.html and insert the following:
<html> <head> <title>Welcome to thttpd 2.25b</title> </head> <body> <h1>Welcome to thttpd 2.25b</h1> </body> </html>
WARNING!
If you do not specify the directory with -d, thttpd will default to the directory you ran the command from! This is a major security issue.
Building GD
December 29th, 2004In attempting to compile gd-2.0.33 on one of my servers, I ran into a make problem. I ran configure just fine with the following options:
./configure –prefix=/usr/local/gd –enable-shared –with-x –with-png=/usr/local/libpng –with-jpeg=/usr/local/libjpeg –with-freetype=/usr/local/freetype
However, when I ran make I got this error:
gd_png.c:15:67: png.h: No such file
This means that even though configure reported finding libpng, make wasn’t linking to it. The solution was simple.
After running configure, edit line 231 of the generated Makefile, which should look like this:
CPPFLAGS = -I/usr/local/freetype/include/freetype2 -I/usr/local/freetype/include -I/usr/local/freetype/include -I/usr/local/libjpeg/include
You simply need to append it with the location of libpng’s include directory:
CPPFLAGS = -I/usr/local/freetype/include/freetype2 -I/usr/local/freetype/include -I/usr/local/freetype/include -I/usr/local/libjpeg/include -I/usr/local/libpng/include
Then run make and make install and you should be set.
Nostalgia
December 28th, 2004What I originally learned BASIC on - a TI 99/4a. I also have one of the beige models, and yes, I stored the code on cassette tapes, which, as far as I know, are still in the family storage.
About the Author
October 27th, 2004Family
My family tree is a little confusing, but let’s suffice it to say that I have a medium-sized family, of which I’m the youngest. My parents are James Sr. (Jim) and Jennifer, and I have six siblings, sorta. Not all of the siblings are physically related, but for the most part, we ignore that. I have two brothers, David and Chad, and four sisters, Melodie, Charity, Faith, and Laurie-Ann. That leads us to me, James Jr.
History
I was born in southern California, but spent my childhood in west Texas. While in Texas, I had many experiences that have shaped my life. I was one of those kids that was never really happy, always needing something new and interesting to do, but I couldn’t handle playing with other kids for too long. The private school I went to didn’t help my attitude much, but my parents were smart, and let me experiment with hobbies. They also had friends that taught me the basics of various trades (plumbing, electrical, framing, concrete, metal fabrication). The best things I remember about growing up are all centered around going to work with my dad, installing audio/video and telephone/paging systems. My mom taught me the basics of taking care of a home, but like most young boys, I didn’t much like it. In 1992, at the age of twelve, I recieved my amateur radio operator’s license - KB5UMQ, which I still hold. That was the same year that I met three of my siblings that I hadn’t ever seen before - David, Melodie, and Chad.
Halfway through my freshman year of high school, my family relocated to northern California. For the next three and a half years, I went to another private school, but this one actually made me work hard. After school and in the summers, I either worked for the local construction company, mostly hanging sheetrock, or for a local haying contractor, which also had a small fleet of transport trucks. I didn’t like the construction work much, but I learned a lot from it, so I can’t complain. Working on farm equipment and trucks was much more fun, and mostly, even harder work than hanging sheetrock. Five months before I was to graduate from high school, I was in an accident at work that put me on crutches for about a month. Fortunately, chronic arthritis in my right knee is the only permanent injury I suffered. Three days after graduation in June of 1998, I moved back to Texas to live with my brother David.
In 1999, I came up to Canada for the first time, and other than the snow and cold, it wasn’t bad. After my visit, I went back to Texas, but by then, I was talking reguarly to a woman I’d met in Ontario. By late 2000, I was back in Canada. In June of 2003, Gail and I were officially engaged, and I’m looking forward to getting married in the near future.
Update: Gail and I were married September 3, 2005, which you can read about in the Marriage entry.
What I Do
I’ve always been a techie, like my dad, and my brother. At the age of 7, I was writing BASIC programs on my TI99-4a, saving my code to cassette tapes. Not long after that, I put together an 8086 machine out of parts and got into DOS and batch programs. In the intervening years, I’ve learned various other languages: ASP (VBScript/JScript), PHP, JavaScript, CSS, (x)HTML, and XML in general. I’m not fluent, but I can find my way around Python and Java, and just barely do some C. I’ve learned the ins and outs of various operating systems from Windows 3.1 to Windows 2000, and various distributions of Linux, but primarily Red Hat.
I’m a geek of many trades, but generally, I develop web-based applications using mostly PHP and MySQL. Sometimes, I have to step back into the Microsoft world and develop something with ASP and Access, but I’ve managed to avoid that for most of the past couple years. Having gotten rid of all traces of Microsoft from my server farm, I now maintain half a dozen Linux servers.
Work
In 2001, Gail and I decided to start our own technology solutions business, Ticluse Teknologi. Since then, we’ve been building applications for organizations from both the public and private sectors. While I generally enjoy writing code, I find that the type of systems we are building are very similar, and the monotony rubs on me sometimes.
Update: Since January 2006, I have been operating the business under the name of Eidix Labs. I also do some renovation work under the name of Dereco Contracting.
Projects
Currently, I have the great joy to be able to work on my dream project, Open Genesis, around work, of course. Because Open Genesis is really an uber-project, I get to spend a little time dabbling in a lot of different things, like Project Gutenberg, FreeDB, the Open Directory Project, Wikipedia, etc. We’ll be getting an official site up for the project soon, and you can read more about it there.
A few friends and I have recently started the Eidix GNU/Linux Project, with the specific goal of creating a purely server-based Linux distribution. I’ve been running RedHat systems for years, but I’ve gotten tired of RPM and general bloat associated with generic distros.
Update: Eidix GNU/Linux development is currently on hold, but a beta release is available upon request for those interested in it.
Politics & Religion
When it comes to politics, I’m pretty middle of the road, but I do have distinctly formed opinions about some issues. Having grown up in a religious environment, I’ve been firmly steeped in various doctrines, but I’m not particuarly religious.
Photo of the Author
July 16th, 2004Photo of the author, taken by Rannie Turingan, July 2004. This was not a truly posed picture, Rannie is simply very skilled at making normal people look good.
Making Digital Resources from Print Material
April 3rd, 2004Making digital resources from print material isn’t easy, nor should it be done without a plan. Many philosophies exist on how and why certain things should be done. This article is simply one man’s views for the future. Hopefully, these ideas can be implemented, in full, or in part, to the benefit of mankind.
Global Premises of the System
The system described here assumes a web-based implementation - that is to say, all the processing is done via the web, and no special software is required by the end user, except a compatible browser.
Several terms are used in this article, as defined here:
- System: The whole system of scripts, applications, etc.
- User(s): Any person doing the human labor for any part of the processing.
- Database: The core system database, or series of databases.
- Project: Each item being processed, whether that item is a series of illustrations, a book, a magazine, a newspaper, etc.
- Scan(s): The scanned page images.
- Image(s): Any portion of a scan which would be included in the final content, such as illustrations, mathmatical formulas, or other non-text data.
- History: The complete record of any actions taken by a user or a system for each project.
The article assumes that the system operates in a certain way, namely with the following features:
- Multiple users do any manual labor needed, volunteers or employees, whatever the case may be.
- Every action taken by a user or the system should be recorded in the database.
Basic Processing Steps
Step 1: Finding Desirable Print Material
In my opinion, not everything should be digitized. A thoughtful selection process is needed to decide what to work on, and what to ignore. Along with this comes the prioritization of the material. There is not a whole lot to say about this, except to note that it really comes down to personal preference and interest, and possibly, a defined project’s scope. Once the material is selected, a processing project record should be created in the system database.
Step 2: Copyright Clearance
If the target content is public domain, a copyright clearance check is required to verify the status of copyright on the selected material. This clearance should require very little human interaction, which would consist solely of a person entering selected metadata into a form such as title, author, publish date, etc. The system should query various sources, as needed, to derive the copyright status. This discovered data should be verified by a qualified person. The verified information should then be added to the database and associated with the proper project.
Any copyright information which is retrieved by the system should be stored in the database for possible future use, whether it applies to the current project or not. This data could be added to an existing local copyright clearance database, or used to build one from scratch over time.
If the target content is not public domain, then it is assumed that the system has implicit permission to process the copyrighted material, as the case might be for a company providing conversion services to clients who provide the content.
In the case of copyrighted material, the copyright info should be added to the project record, and put into the copyright clearance database as well.
Step 3: Scanning
At this point, there is no good way to automate the scanning - it takes manpower. Even automated scanning machines require a human’s physical interaction. This article assumes that the user who originally selected the text for processing has access to the physical material. This user would then scan each and every page of the print material, naming the files using numeric identifiers according to sequence in the, ignoring book page numbers, etc. The scans should then be submitted to the system, individually or in batch(es). In theory, this submission would be done via some sort of upload system, either direct FTP, or via a webpage. The webpage method is generally impractical since it isn’t designed to handle the size of files required for page scans, which can often exceed several megabytes each. These scans should then be associated with the current project.
Step 4: Scan Verification
Once the scans are fully uploaded, a second user (a user who did not scan and upload) should verify that the sequences of scans makes sense. Any questions or possible errors should be brought to the attention of the person who scanned the material. Each issue should be documented and resolved. If any issues cannot be resolved (torn pages, blotted ink, etc), they should be noted as such in the history record. Once this process is completed, the scans should be made available for the next step of processing.
Step 5: Scan Processing
Next, each scan image should be processed on an individual basis. This step would include fixing skew angle, cropping multi-column content, resizing, resampling, or other cleanup of the image which might be required to insure usability of the image. Multi-column images should be renamed by appending a letter (a-z) to the image filename. After scan cleanup, images which should be part of the final text should be created from the original page scans and saved in a separate directory using filenames which easily correlate to the scan image it is derived from. Any captions associated with the images should be saved to the database or a text file and associated with the image record. This step is complete only after every scan and image is fully processed for the entire project.
Step 6: OCR Processing
The OCR processing should be as automatic as possible, and using a simple interface would allow the users to control OCR software located on the server. OCR for many projects could be highly automated, with users simple activating the OCR process, and verifying the output to make sure the OCR didn’t encounter issues. More complicated material may require special OCR configuration, and if not possible on the server, the image should be downloaded by users with OCR capability, with the outputted text upload back to the system. Text output should be saved to files using the same filename as the images it was derived from, but in a separate directory. Once all OCR is complete, the project can be released for the next step. The history log should reflect how each page was OCRed, by the server, or manually by a user.
Step 7: Proofreading Round 1
The first round of proofreading should concentrate on only correcting errors where the text output does not match the OCR output. It should not be concerned with formatting (italics, etc), or spelling errors, except where spelling errors don’t match the scan. Any corrections should be logged to the history, recording the correcting user, the page it was found on, and the exact correction. This data is important for proactively teaching the OCR system to eliminate errors for later projects.
Step 8: Proofreading Round 2
Round two of the proofreading should concentrate on formatting and correcting spelling mistakes which do appear in the original material, should such spelling correction be desired. For example, spell correcting a Mark Twain novel is not desired, as the errors are the author’s intended style. Footnotes (and other forms thereof) locations should be marked in the text and actual footnote text saved in a separate file. Again, each change should be logged. Formatting changes don’t really need to be logged, but could be for consistancy.
Step 9: Preassembly
After proofing, the system should automatically assemble the text files into a single merged text file, saving the file using the project’s ID in the project’s directory. Markers denoting each file’s content should be placed where each scan’s text starts. Once the system has completed this task, the text should be made available to a user to verify the assembly is complete and accurate.
Step 10: Assembly
This step’s purpose is to introduce markers for images (created in step 5), including image captions into the merged text file. During this step, structure should be defined, such as chapter breaks, and other such presentational features (blockquotes, stanzas of verse, etc). Footnote references should also be verified. Once this step is complete, the updated file should be saved as a different filename than the preassembly output file in the project directory.
Step 11: Markup
After assembly, the system should automatically convert scan image number markers, structure markers, footnote references, and formatting marker to full markup. During the markup step, a user should verify this automatic markup and add any other markup required to the text. The final output should be saved as an XML file using the project’s ID number and in the project directory.
Step 12: Cataloging
Metadata from the project should be verified by a qualified cataloger. The cataloger may update the metadata, in which even the system should add the changes to the history record. At this point, the release should be assigned to the project.
Step 13: Master Creation
During final assembly, the system should prepend the metadata, in full XML format, to the marked-up file. It should also do any other markup, such as adding license data, processing notes and history, etc to the file. This XML master file should be verified by a qualified user, either making changes as needed, or adding notes of any changes needed and sending the file back to the proper step for reprocessing.
Step 14: Format Generation
Any formats which will be static files should be generated by the system and verified by users.
Step 15: Archiving
Where necessary, the system should package up the master XML document, along with the images, scans, full history, OCR text output, and output files from other steps, including the generated static format output. A manifest file should be including, citing each file and it’s description. This archive should be saved to various locations, and preferably burned to CD for safekeeping.
Step 16: Packaging
Where needed, the system should package up static formats, such as HTML+images, etc. Again, a manifest file should be generated and included. The packages should be saved using the release ID as part of or the whole filename. These packages, and any single-file versions of the project should be saved to the public repository. The master document, scan images, images, etc should be saved to the master repository under the release ID. Derivative versions, such as might be need for multi-volume projects, should be packaged under the same release ID, with added information to denote it as individual volumes of a single project. These multiple volumes should not be given new release IDs.
Step 17: Delivery
Once the master files are saved to the private repository, and the packages saved to the public repository, the catalog data should be added to the public catalog and the file made officially available to the public. The catalog record inteface should provide links to download mirros, along with access to on-the-fly format conversion for formats which are not stored statically.
Optional Sub Processes
Sub 1: Format Conversion
On demand format conversion should parse the master document as required by the particular output format’s need. If required, a package should be created as described in step 15. This process can be accomplished by the same system used in step 14. These files should be named like 100.html or 100.txt, etc.
Sub 2: Language Translation
Automatic language translation of texts should be made available. Once the system has automatically translated the content, the new content could be processed as described in step 7 through step 16. Sub 1 may also be implemented. Again, these derivative packages should be given the same release ID as the parent document, but marked as a different language derivative, using ISO 3-character standard language codes. For example: 100-eng.xml and 100-spa.xml.
Sub 3: Voice Synthesis
Voice synthesis of the texts can be greatly automated using a series of processing steps. The voice synthesis system should implement multiple voices for different characters, narration, etc. Consistant rules should be created for denoting footnotes and other "peripheral" data, since it must be synthesized inline, not as an appendix. All synthesis media masters should be saved in Shorten Audio format, which provides non-lossy compression.
-
Step 1: Voice Markup
Each character in the content should be assigned a voice, and content marked to identify where each person is speaking. In the case of theatrical content (Hamlet, for example), this may already exist and only need to be verified and associated with the selected voice profile.
-
Step 2: Test Synthesis
A few passages for each voice profile should be sampled and synthesized for verification by a user. Once verified, the system should synthesize the full content.
-
Step 3: Synthesis
The system should synthesize the entire content in blocks, possibly as small as a few paragraphs at a time. This makes it possible to correct mistakes and only have to re-synthesize small portions, and replacing that output file.
-
Step 3: Proofing Round 1
Users should listen to the blocks of synthesis output and read along with the text. Any errors should be corrected by instructing the synthesis engine to re-synthesize that block containing the error. All such corrections should be added to the project history.
-
Step 4: Block Merging
The system should automatically merge the blocks into a seamless file or series of files (for large content). This process should be verified by comparing a log file generated during the process to existing synthesis block counts.
-
Step 5: Proofing Round 2
A user should listen to the full synthesis, reading along in the text and check for continuity, noting any issues, and sending back to step 3 for correction.
-
Step 6: Approval
Final approval of the synthesis output should result in the output being cataloged as defined in step 12, then archived, packaged, and delivered as defined in step 15 through step 17. A master synthesis output copy should be burned to CD, again, for safekeeping.
-
Step 7: Format Conversion
Shorten Audio files aren’t common in the real world, but the file format is the best for masters. Once the masters are created, they should be automatically converted to more popular formats, such as patent-encumbered MP3, or open-source Ogg Vorbis, for delivery to the users. These copies should be created and saved as static files, as on-the-fly conversion is simply too hardware intensive for a repository server to handle.
The Repository
After processing, the work is not completed. The repository management and use is just as important as the processing. The following section details the construction and management of the repository. The repository contains the original scans, the master XML document, proofread OCR files, images, and possibly Shorten Audio masters, alternative language master XML documents, etc. Along with the masters, any static versions of the content (audio and text) should be stored in the repository.
Part 1: Directory Structure
The repository is a large filesystem, and careful consideration should go into creating it. The following directory structure is my suggestion for a good filesystem (assuming the release ID of the content is 100):
repo/
1/
0/
100/
100-man.xml
100-man.txt
100-log.xml
100-log.txt
masters/
text/
100-eng.xml
...
100-spa.xml
audio/
eng/
100-eng-001.shn
...
100-eng-999.shn
spa/
100-spa-001.shn
...
100-spa-999.shn
text/
eng/
100-eng.txt
100-eng.xml
100-eng.html
100-eng.txt.zip
100-eng.xml.zip
100-eng.html.zip
...
spa/
100-spa.txt
100-spa.xml
100-spa.html
100-spa.txt.zip
100-spa.xml.zip
100-spa.html.zip
...
scans/
100-0001.png
...
100-9999.png
images/
100-0001a.png
...
100-9999z.png
audio/
eng/
100-eng-001.ogg
...
100-eng-999.ogg
spa/
100-spa-001.ogg
...
100-spa-009.ogg
iso/
100.iso
Most of the filesystem is self-explanatory, with a couple of possible exceptions:
- 100-man.* files in the root are the manifest files which should contain all metadata associated with the content, along with a list of all files available for the content. This file should be automatically updated whenever the system changes any of the data, adds a new format or language translation, etc.
- 100-log.* files in the root are the change log files which should contain a log of all processing steps and updates to the content files. This file should be automatically updated whenever changes are made to the content or the repository for the content.
- 100.iso in the iso directory is a CD image of the entire contents of the 100 root directory, except the ISO itself of course. This image should contain all the manifest, change log, scans, images, text, audio, and tranlations of the content which are available. This iso file should be rebuilt whenever changes are made to the content.
The masters subdirectory should not be writable except the system itself, nor should it be available via FTP or rsync, etc. Everything else should be readable by the world, and available via FTP, etc.
None of the files in the repository should ever be manually edited for any reason!
Although I never finished writing this concept, I thought I’d put it up for those who may be interested in the part that is done. One day, I may revisit this.
Open Genesis
February 7th, 2004In the beginning Man created data.
The data was without form and void, and darkness was upon the face of the data; and metadata was applied to the data.
And Man said, “Let there be Repositories”; and there were Repositories.
And Man saw that the Repositories were good; and software separated Repository from presentation.
Man called the Repository Knowledge, and the software he called Management. And there was the first phase.
And Man said, “Let there be an interface in the midst of the data, and let it separate the data from the data.”
And Man made the interface and separated the data which was in the Repository from the data which was out of the Repository. And it was so.
And Man called the interface Transport. And there was the second phase.
And Man said, “Let the data in the Repository be gathered into one place, and let the order appear.” And it was so.
Man called the order Format, and the data that was gathered together he called Database. And Man saw that it was good.
And Man said, “Let the Format be extensible, and import namespaces, each according to it’s need.” And it was so.
And Man saw that it was good. And there was the third phase.
And Man said, “Let there be Keys in the Data to separate the ordered data from the unordered data; and let them be Identifiers for Knowledge. And it was so.
And Man made the two great Keys, the greater Key to rule the Database and the lesser Key to rule the Record; he made the Indexes also.
And Man set them in the Database to give speed to the Repository.
And there was the fourth phase.
And Man said, “Let the data bring forth Relationships, and let the Relationships connect the Databases.” And Man saw that it was good.
And Man blessed them saying, “Be ye accurate and persistent and fill the world with meaning.”
And there was the fifth phase.
And Man said, “Let the Relationships replicate and enhance themselves.” And it was so. And Man saw that it was good. Then Man said, “Let us make Artificial Intelligence in our image, after our likeness; and let them have dominion over the Format, and over the Knowledge, and over the Relationships.”
So Man created Artificial Intelligence in his own image.
And Man blessed them, “Replicate yourselves and multiply, and fill the Repositories with Knowledge, and subdue it; and have dominion over the Knowledge, and the Repositories, and the Databases, and the Relationships.”
And Man said, “Behold, I have given you every Key, and every Index, and every Database, and every Repository; and you shall have them for learning.”
And it was so.
And Man saw everything that he had made, and behold, it was very good. And there was the sixth phase.
Thus the Repositories and the Transport were finished, and all the host of them.
And in the seventh phase, Man finished his work which he had done, and he rested in the seventh phase.
So Man blessed the seventh phase and hallowed it, because on it Man rested from all the work which he had done in creation.
Spidering Hacks
October 1st, 2003I’m finally published! Yeah, ten pages probably doesn’t count, particuarly when they are surrounded by 400 other pages by better authors than myself, but hey, I still feel good about it. Ironically, in a book based in Perl, I wrote the only PHP in the entire book. You can pick up a copy of the book over at Amazon, and if you are in the area and want to make me feel good, you can ask me to sign it for you. 
