Token-Flag (Optional)

The token-flag in the data scanning parameters part of the Load Definition File is a string used to subdivide a document into sections associated with tokens for storage and indexing.

The document can be subdivided by inserting the token-flag into a comment where the comment text is normally placed. To associate the subdivision of the document with a token, the token-flag must be followed immediately by the token within the comment, without any intervening blanks. The section of the document associated with the token continues until the next token-flag is encountered -- even if no token is associated with that token-flag.

Example Token-Flag = section:

<!-- section: intro introductory portion of article -->

As the end of the current session of Congress draws

. . .

. . .

<!-- section: end of introductory portion of article -->