A Package in a Load Definition File is a collection of records. A package is the major subdivision of the data, and a component in defining indexes. An Index is created over one or two data fields within all records belonging to a defined set of packages. In addition, packages work in conjunction with primary indexes to organize the storage of both data and nonprimary indexes. In that way, how packages are defined can effect both transferring of data to the search engine, and the searching of the data by the end user.
The data-store can contain large amounts of data. And, the associated indexes can contain even greater amounts of data — far greater. Often, data stored within the same load can come from a number of different sources. To make it easier for the developer to organize the large amount of data and various sources, Flexible Search employs a system of packages and sections.
Packages are simply groups of sections, which are used to facilitate entering data into the system. While sections have no direct effect on the organization of the data-store or indexes, grouping sections into packages directly influences the organization of both the data-store and indexes.
Each package within the load is identified by a unique name and contains one or more section definitions.
Two types of sections are supported by Flexible Search:
Used to provide processing information for data provided via a Data Source File. The section definition contains information used to locate the Data Source Identifier File and DataSource File. It also contains the information needed to properly extract data from the Data Source File and process it correctly. There may be any number of data file sections within a package.
Used to provide processing information for data provided by scanning HTML, PDF, office or text documents. The section definition contains information used to locate HTML documents and to properly extract the data from those documents.