Document Information
Author
Paul Klink
Copyright
© Paul Klink 2007
Release Date
4 Nov 2007
Status
Early Draft
Document Version
0.6
Fielded Text Standard Version
1.0
Change Information
Introduction
Definition
Fielded Text is a standard which provides structure and schema definition to text files which contain tables of values. Typically, a line in these text files represents a record (or row) and the line itself is divided into fields (or columns). Comma Separated Variable (CSV) files and Tab Delimited files are examples of Fielded Text Files.
Purpose
The purposes of the Fielded Text standard are:
To make it easier to parse files containing fielded information (ie tables of values).
To assist in generating files which conform to a specified format.
To increase the chances of the information in a file being correctly interpreted.
Improve methods of transferring and exchanging tables of data
Allow files containing fielded information to be accessed with database like techniques.
Compatibility
The Fielded Text standard aims to be compatible with nearly all existing files containing fielded information. It supports the following capabilities to provide compatibility with existing files:
Fields separated by a delimiter character
Fixed Length Fields
Mixed Fixed Length and Delimiter separated fields in a line
Quoted Fields (Optional or Explicit)
New Lines in Quotes
Automatic New Line detection or specified New Line character
Comments
Ignoring Blank Lines
Ignoring extra characters/fields in lines
Handling language cultures
Character Encodings
Multiple Heading Lines (both delimited and fixed length)
Substitutions (escape sequences)
Embedded (Stuffed) Quote Characters
Boolean, DateTime, Decimal, Float, Integer and String fields
Field Heading Constraints
Null fields
Constant fields
Specify format and styles of fields
Lines can have different sequences of fields based on the value of “key” fields
Fielded Text Meta
The “Meta” of a text file specifies the format of a fielded text file. It specifies items such as
What character is used to delimit fields
The format and type of each field
The substitutions used in the file
etc
The Meta is specified in an XML File/Stream. The schema for this XML file is specified in this document.
Files and Streams
In order to make this standard more readable, it refers to Fielded Text “files” and Fielded Text Meta “files”. However, in most cases, “file” can be respectively construed as any content or stream of data containing fielded information or XML data which is Meta for fielded text. For example, it can also represent: web pages, memory streams, other IO streams etc.
Declared and Undeclared Fielded Text files
Fielded Text files can either be declared or not declared. Declared Fielded Text Files begin with 2 lines which contain the following information:
A special string which identifies the file as being a Fielded Text file.
The version of the Fielded Text Standard which this file conforms to.
A link to the Meta which specifies the format of this Fielded Text File
Custom/File specific information
Undeclared Fielded Text files do not begin with these two lines and do not contain the information. When these files are parsed, then Meta file needs to be explicitly specified. Existing CSV or Tab Delimited or other text files containing fielded information are undeclared Fielded Text files.
Culture and Character Encoding
Culture
Different countries and regions throughout the world have different conventions concerning how to display dates and numbers and how to compare text.
The “Culture” Meta property specifies which regional conventions should be used.
Whitespace
Whitespace Characters are printable characters which provide horizontal and vertical separation and do not provide a visual mark. The selected culture determines whether a character is a whitespace character however the following characters are always considered as whitespace:
Horizontal Tab (ASCII 9)
Line Feed (ASCII 10)
Vertical Tab (ASCII 11)
Form Feed (ASCII 12)
Carriage Return (ASCII 13)
Space (ASCII 32)
Whitespace characters can have special significance in certain contexts within a Fielded Text file. For example, a Line Feed character may mark the end of a record. The special uses of whitespace characters will be further covered throughout the rest of this document.
Lines
A Fielded Text file consists of 0 or more lines. How a line starts and ends, and the types of lines in a fielded text file is described below.
Start and End of a Line
A line starts with either the first character in a file or the first character after the End of Line (EoL) character(s) that flagged the end of the previous line. A line ends with the first EoL character(s) encountered after the start of the line. The last line in the file can either end when the the EoL characters are encountered or when the end of the file (EoF) is reached. The EoL character(s) are not considered part of the line.
The “EndOfLineType” Meta property determines which character(s) are the End of Line character(s). This property can have one of three values:
“Auto”
Auto specifies that any of the following 3 character sequences will flag the end of a line:
<Carriage Return> (ASCII 13)
<Line Feed> (ASCII 10)
<Carriage Return><Line Feed> (ASCII 13 followed by ASCII 10)
When generating files, the “EndOfLineAutoWriteType” Meta property specifies which character(s) should be used to mark the end of a line if the “EndOfLineType” is “Auto”. The “EndOfLineAutoWriteType” property can have one of the following values:
“CrLf”
The EoL characters are <Carriage Return><Line Feed> (ASCII 13 followed by ASCII 10).
“Cr”
The EoL character is a <Carriage Return> (ASCII 13).
“Lf”
The EoL character is a <Line Feed> (ASCII 10).
“Local”
One of the the above EoL character sequences is used based on which is used in the local environment / operating system. This value ensures that any Fielded Text files generated on a computer can be correctly viewed with other programs on that same computer.
“Char”
Char specifies that the EoL character is specified by the “EndOfLineChar” Meta property. The “EndOfLineChar” property must contain a string with one character only in it. This character becomes the EoL character.
“CrLf”
CrLf specifies that the character sequence <Carriage Return><Line Feed> (ASCII 13 followed by ASCII 10) flags the end of a line.
Line Types
A Fielded Text file contains 4 types of lines
Heading Line
A Heading line is a line in the Heading Section of a Fielded Text file. These lines correspond to Headings often placed at the top of files containing fielded information.
Record Lines
These lines only exist in the Body Part. They contain the actual fielded information. Typically, one line represents one record (or row) of fielded information. However if a field contains EoL character(s) (and the “AllowEndOfLineCharInQuotes” Meta Property is True), then it is possible for a record to span multiple lines.
Comment Lines
Fielded Text files can contain comment lines. Any line which is not a Heading Line or a Record Line or an ignored Blank Line (see below), needs to be a comment line. This allows Fielded Text files to be easily parsed by other types of Parsers.
A Comment line is a line which begin withs the “Comment Character”. (There are possible 2 exceptions to this, Heading Lines and New Lines embedded in a quoted field; these are covered later in this document). The “Comment Character” is either specified by the Meta or is specified by the Declaration part (if it is present).
Comment lines are generally ignored in a Fielded Text file however in 2 sections (Declaration and Embedded Meta) they are not ignored.
Ignored Blank Lines
Blank lines are treated like any other non-comment line if the “IgnoreBlankLines” Meta Property is False. However if this Meta Property is True, then Blank Lines become Ignored Blank Lines and are ignored. (Similarly to Comment Lines, there are possible 2 exceptions to this, Heading Lines and New Lines embedded in a quoted field; these are covered later in this document).
Structure
Overall
At the highest structural level, a fielded text file consists of 2 parts: Header and Body. Both parts are optional. In fact an empty file can be a valid Fielded Text file. The Header part always comes first and is immediately followed by the Body part.
Header Part
The Header itself consists of up to five sections. These are:
Declaration
Comments
Embedded Meta
Comments
Heading Lines
Each of these sections are optional. If they are present, it will be in the above order.
The Declaration section, if it is present, will be the first 2 lines in the file. The first character in the first line of the declaration becomes the “Comment character”. This overrides the Comment Character specified in the Meta. Each of these 2 lines in the declaration part begin with this comment character.
If the declaration specifies that the Meta should be loaded from either an external file or an external URL, then this Meta will be downloaded immediately after the 2 declaration lines. The new Meta takes affect for the 3rd line onwards. If the new Meta specifies a different Comment character, then this also will take affect from the 3rd line onwards.
The Comments and Embedded Meta sections can contain comment lines or ignored blank lines. Both Comments sections are ignored when parsing.
The Embedded Meta section will be present if the Declaration specifies that the Meta for this Fielded Text file is embedded within the file. This part contains the XML which is the Meta for this Fielded Text file. The lines containing a standard XML stream (conforming to the Fielded Text Meta Schema) however each line begins with the comment character and optionally followed by some white space characters. The start and end of this section is determined by the detection of specific XML tags/elements.
If an Embedded Meta is present, then this new Meta will take affect in the line that immediately follows the Embedded Meta section. If the new Meta specifies a different Comment character, then this also will take affect after the Embedded Meta section.
The number of Heading Lines contained in the header is specified by the “HeadingLineCount” property. If there is one or more Heading Line, then the first Heading Line will be the first line that is not a comment line or an ignored blank line.
The remaining Heading Lines will immediately follow the first heading line. If any of these remaining Heading Lines are blank, then they are still considered to be heading lines – even if “IgnoreBlankLines” Meta property is True. Also, if any of the remaining Heading Lines begin with a comment character they are also still considered to be heading lines (and not comments). In other words, the Heading Lines are always consecutive.
If the Meta specifies that there are no Heading Lines then the Body part starts where the Heading Lines part would have started.
Declaration Section
Signature and Comment Character
A Declaration Section is contained within a Fielded Text file if the following 2 conditions are met:
The file contains at least 17 characters, and
The 2nd to 17th characters match the Fielded Text Signature string.
The signature string is “|>Fielded Text<|” (not including the surrounding quote characters and with a space character between “Field” and “Text”).
If the above conditions are met, then the “Comment Character” is set to the first character in the file.
Declaration Parameters
The rest of the first line and all of the second line consist of Declaration Parameters.
A Declaration Parameter is a name/value pair. The name/value pairs have a format of: <name>=<value>. The value part of the name/value part is a string which is always encoded with standardised Quoting (see section X). The name part can be any string provided it does not contain the “=” character and it cannot start or end with a white space character. The name part is not case sensitive. The name and value parts are separated by an “=”character. Any white space characters between the “=” character and the name or the value, are ignored.
Declaration Parameters themselves are separated by one or more white space characters. The first declaration parameter can either immediately follow the Signature string or be separated from the Signature string by one or more white space characters.
Version Declaration Parameter
The first declaration parameter must always be the Version declaration parameter. This parameter must always be present in a Declaration. (No other parameters are compulsory.) The Version parameter specifies the version of the Fielded Text standard which a Fielded Text file conforms to.
The name of this parameter is: “Version”.
The value of the Version parameter has a format of:
<Major
Version Number>.<Minor Version Number>[.[Any string not
containing white space characters]]
The “Major Version
Number” and “Minor Version Number” must always
be present. The “Minor Version Number” can optionally
be followed by a period and then (optionally again), a string.
The string cannot contain white space characters.
A Fielded Text parser should use the version parameter to determine if it supports the version of the Fielded Text standard a file conforms to.
Meta Reference Declaration Parameters
A Meta Reference Declaration Parameter can be included to associate a Fielded Text file to its Meta. There are 3 different types of Meta Reference Declaration Parameters:
MetaEmbedded parameter (name: “MetaEmbedded)
If this parameter exists, then the Meta for the Fielded Text file is embedded within the Fielded Text file. The value is ignored however it is recommended that the value be set to True.
MetaFile parameter (name: “MetaFile”)
The value for the MetaFile is a path of a file which contains the Meta for the Fielded Text file.
MetaUrl parameter (name: “MetaUrl”)
The value for the MetaUrl parameter is a URL where the Meta for the Fielded Text file can be downloaded.
Only one of the above 3 parameters should be included in the declaration. If more than one are included, then the one with the highest priority according to above list is to be used (1 = highest). If more than one of the same type (and highest priority) are included then the first of these in the file should be used.
Custom Declaration Parameters
Custom or user specific declaration parameters can be added to the declaration. The name of custom declaration parameters should not conflict with any existing declaration parameters defined by this standard. To avoid conflict with declaration parameters defined in future versions of this standard, custom parameters should have names beginning with “x-” or “X-” (not including quotation characters).
Embedded Meta Section
The Embedded Meta section only exists in a file if the declaration section contains the “MetaEmbedded” parameter. If this parameter is not present then even if lines representing the Meta are present, then they should be ignored (treated as comments). If the parameter is present, then the Embedded Meta section must be present otherwise the file is invalid.
All lines in the Embedded Meta section are comment lines. However if you remove the comment character that starts each line, then the lines make up the XML that represents the Meta for the file. After the comment characters have been removed from the lines, the resulting XML can be processed by any standard XML reader to get access to the Meta properties and elements.
The Embedded Meta Section is considered to be present if the string “<FieldedText” (not including quotes and case sensitive) is detected in a comment line in the header (but not in the declaration).
If the Embedded Meta section is determined to be present, then the start of the Embedded Meta section is determined as follows:
If a line prior to the line containing “<FieldedText”, contains the string “<?xml ” (not including quotes; one space character at the end; and case sensitive) then the Embedded Meta section starts from this prior line. The prior line cannot be in the declaration section.
If no prior line contains the string “<?xml ”, then the Embedded Meta section starts from the line containing the “<FieldedText” string.
The end of the Embedded Meta section is detected by checking if a line contains the string “</FieldedText>” (not including quotes and case sensitive). The line that contains this string is the last line of the Embedded Meta section.
The XML which makes up the Meta starts from either:
The string “<?xml ” if it exists on the first line and is before the “<FieldedText” string.
Otherwise the XML starts from the “<FieldedText” string on the firstline.
All characters on the first line before the start of the XML are ignored.
All characters after the “</FieldedText>” string on the last line are also ignored.
All lines in the Embedded Meta section must be comment lines. If any line in this section does not begin with a comment character, then the file is invalid.
Comments Sections
Both comments sections only contain comment lines. These lines are always ignored when parsing.
Heading Lines Section
Heading Lines contain fielded data. However the values in these lines are the headings for the fields. Each field has an array of heading values. These values can describe the contents and purpose of a field or they can provide other supplementary information about the field.
The number of headings in a field is the same as the number of Heading lines. Both counts are therefore specified by the “HeadingLineCount” Meta property. The values in each successive heading line corresponds to equivalent indexed values in the field's array of headings.
If one or more of the heading lines is specified, then it is also necessary to specify which of these lines contains the “Main” heading. The “MainHeadingLineIndex” Meta property specifies the index of the main heading line. For example, if the first line held the main heading, then “MainHeadingLineIndex” would be set to 0. The main heading line is used in conjunction with Heading Constraints. These are described in Section X.
Body Part
The Body Part starts immediately after the Heading Lines section, or if there are no Heading Lines, where the Heading Line section would have started.
This part contains 3 types of lines:
Record Lines
Comment Lines
Ignored Blank Lines
Comment Lines and Ignored Blank lines within the Body are ignored when parsing.
Record Lines contain the fielded information. Normally one line represents one record (or row). However it is possible for a record to span multiple lines. This will occur if the EoL character(s) are embedded in a field. In this case, EoL characters do not designate the end of a record and are treated like other characters. EoL characters are allowed to be embedded in a field if the following 2 conditions are met:
The “AllowEndOfLineCharInQuotes” Meta property is True, and
The field is quoted. That is, the field is surrounded by quote characters.
If an EoL characters are embedded in a field, then the subsequent line is a record line – even if it starts with a comment character or it is blank line. Comment Lines and Ignored Blank Lines cannot occur within a record that spans multiple lines.
Record
Records contain the fielded data in a fielded text file. Each record contains an ordered set of values which correspond to the fields and sequences defined in the Meta. Typically a record corresponds to one line in the fielded text however it can span multiple lines as described above.
The Meta defines the Fields which a record can contain. The order of the fields in a record is defined by the Sequences in the Meta. If no sequences are defined in the Meta, then an implicit sequence is used. The implicit sequence consists of the list of all fields in the order they are declared in the Meta.
Each value in a record must conform to the properties of its corresponding field in the record. If there is only one sequence or the implicit sequence is used, then the list of fields will be the same for all records. If there is more than one sequence in the Meta, then it is possible for records to have different lists of fields. This is further described in the Sequences section.
Fields can either be “Fixed Length” or “Delimited”. If a field is “Fixed Length”, then the subsequent field begins immediately after the last character in this field. If a field is “Delimited”, then the Delimiter Character (specified by the “DelimiterChar” Meta property) determines when the field ends. The first Delimiter character after the start of the field, which is not embedded in the field, flags the end of the field. The last character of the field is the one prior to the delimiter character. The subsequent field starts immediately after the delimiter character.
The “IgnoreExtraChars” Meta property determines if any extra characters can occur in a line after the last field in a record. If this property is False, then the EoL character(s) must immediately occur after the last field in a record. However, if it is True, then all characters after the last field are ignored until the EoL character(s).
If the last field in a line is a delimited field, then if the “IgnoreExtraChars” Meta property is False, then the Delimiter Char must not end this field. Since it is the last field in a line, the EoL characters will end this field. If the “IgnoreExtraChars” Meta property is True, then the last field can be ended by either the Delimiter Char or by the EoL character(s).
The last field in the last line of a file can also be ended by the End Of File (EoF). If the last line does not end with the EoL character(s), then the EoF places the same role as the EoL in ending the last field.
Order of fields in a Record
The order of fields in a record is determined by sequences. Sequences are described in section X. However a Meta may not specify any sequences. In this case the order of fields in a record is determined by the “implicit sequence”. The implicit sequence is itself determined from the resolved order of fields.
The order of the Field elements in the Meta, and
The value of the “Index” property in a field Element.
The order of the elements in the Meta determines the implicit order, however field's “Index” property can override the field's position in the implicit order. Section X details how a resolved order is determined from a list of elements.
The resolved order of fields forms the implicit sequence. If there are no sequences, then the fields in a record will be in the order specified by the implicit sequence.
Field
A field specifies the type of data a value in a record contains. It also specifies how that data is formatted. Each field is defined in the Meta with a “Field” element.
Field Name
Each field has a name. The “Name” field Meta property specifies a field's name. Names do not have to be unique within a Meta however it is recommended that they do. Field names are also not case sensitive. When searching for a field by name, the search should be case insensitive and use the culture specified by the Meta. Also note that a field's name can be a blank string.
The main heading in a field can also be a field's name. If the “NameIsMain” heading constraint is selected then:
When parsing a fielded text file, the field name is set to the value in the field's main heading.
When generating a fielded text file, a field's name is used as the field's main heading.
Field Id
Each field has an “Id” Meta property. This property holds a 32 bit signed integer. The purpose of an Id is to allow fields to be identified via an integer instead of a name. This makes it easier to use fields in switch/case statements. However the Id can be used for other purposes.
Headings
The Meta can contain the heading values expected for a field. The headings themselves are in the Fielded Text file however these values can be validated against the heading values stored in the Meta. A field has as many headings as specified by the “HeadingLineCount” Meta property, however the Meta does not need to store that many headings. If less are stored in the Meta, then the remaining headings are set to empty strings and if more are stored, then the excess are ignored.
Heading Constraint
A field's “HeadingConstraint” property specifies how the a fields headings in the Fielded Text file should be validated when the Fielded Text file is parsed. This property can have one of 5 values:
“None”: No validation.
“AllConstant”: The field's headings in the Fielded Text file must match the heading values for that field stored in the Meta. The comparison is case sensitive.
“MainConstant”: The field's main heading in the Fielded Text file must match the corresponding heading value for that field stored in the Meta. The comparison is case sensitive.
“NameConstant”: The field's main heading in the Fielded Text file must match field's name. The comparison is case insensitive and uses the culture specified by the Meta.
“NameIsMain”: If this constraint is selected, then, when a Fielded Text file is parsed, the field's name is set to the field's main heading value. When a file is generated, the field's main heading is set to the field's name. This constraint allows field names to be dynamically assigned directly from the text file.
Field Text, Raw Value and Value
The data stored in a field for a record can be interpreted in several ways. This document refers to this data in the following ways:
Field Text
The Field Text is the complete string for a field as it exists in the file. For delimited fields, the string includes quote characters where fields are quoted and any whitespace around the quote characters. It does not include the delimiter character.
Raw Value
The Raw Value is the string for a field after it has been processed for quotes and substitutions. It is the string value which will be cast to the data type corresponding to the field's data type (eg. Integer, DateTime). When writing a text file, the Raw Value is the string that is obtained after converting the field's value to a string and then (when necessary) applying padding and truncation. For Fixed Width fields, the Raw Value is always the same as its Field Text.
Value
A field's data as a type corresponding to the field's “DataType” property.
Constant Fields
The “Constant” field Meta property specifies whether a field is constant. A Constant field has its value defined in the Meta. When a Fielded Text file is parsed, each value in the file for that field, must match the constant specified in the Meta. If it does not match, then a parsing error occurs. When a file is generated, the constant value specified in the Meta is always written out to the fielded Text file.
The constant value itself is stored in the “Value” field Meta property. This property is ignored it the “Constant” property is not True. The values themselves are stored in this property using their respective standardised format. That is, the culture and the field's format and style properties do not affect how data is stored in the “Value” property. Section X describes the standardised formats for all of the field data types.
A Field can also specify the constant to be Null. If the field is constant, and the “Null” meta field property is True then the field value must always be Null. See the section on Field Values for a definition when a field value is considered to be Null. Note that the “Null” meta property is ignored if the field is not constant. Also note that “Null” field meta priority takes priority over the “Value” field meta property.
Delimited Fields
If the “FixedWidth” field Meta property is False or not present, then the field is a Delimited field. Delimited fields can be of variable length and they are terminated with the first non-embedded Delimiter Character after the start of the field (or the EoL character(s) if the field is the last in the record).
A Delimited field can either be quoted or not quoted. If a field is quoted, then the first non-whitespace character in the field must be the Quote character which is specified by the Meta “QuoteChar” property. Also the last non-whitespace character prior to the terminating delimiter character, must be the Quote character. The characters between the enclosing quote characters will make up the Raw Value of a field (after being processed for embedded quotes and substitutions).
Any Delimiter characters between the enclosing quotes of a quoted field are not treated as delimiters. They are simply treated as characters in the raw field value.
The Quote character itself can be included within a quoted field if the “StuffedEmbeddedQuotes” Meta property is True. In this case, if there are 2 successive quote characters within a quoted field, then neither of these 2 characters is the ending quote character. Instead this pair of quotes represents one normal quote character within the raw value of the field. If the “StuffedEmbeddedQuotes” Meta property is False or not present, then the first quote of a successive pair would be treated as the end enclosing quote character for the field.
Quoted fields also allow EoL character(s) to be included in the raw value for a field. If the “AllowEndOfLineCharInQuotes” Meta property is True, then any EoL character(s) which appear within a quoted field, do not flag the end of a record. Instead they are treated like normal characters and are part of the raw value of the field.
If a field is not quoted, then all the characters from the first to the one prior to the terminating delimiter character, are part of the raw value of the field (after being processed for substitutions). The “StuffedEmbeddedQuotes” and the “AllowEndOfLineCharInQuotes” Meta properties do not change the behaviour of any of the characters in a field that is not quoted.
The “ValueQuotedType” field Meta property determines whether a field in a record is quoted. This property can have one of three values:
“Never”
Specifies that the field is never quoted.
“Always”
Specifies that the field will be quoted unless it has a Null value. (Fields with a Null value are never quoted).
“Optional”
When parsing a text file, if the first non-whitespace character is the Quote character, then the field is quoted. Otherwise it is not quoted.
When writing a text file a field will be quoted if the field is not Null and any of the following conditions are met:
The “ValueAlwaysWriteOptionalQuote” field Meta property is True.
The the first non-whitespace character in the Raw value (the text that represents the field's value) is the Quote Character.
The “ValueWritePrefixSpace” field Meta property is True and the first character in the Raw value is a whitespace character.
The Raw value contains the Delimiter character.
The Raw value contains any of the EoL character(s).
When generating a fielded text field, you can use the “ValueWritePrefixSpace” field Meta property to place an extra space in front of a delimited field. This visually increases the separation between the fields and may make the field easier to review. Prefixing a space can be done on a field by field basis and is activated by setting this field property to True. If a field has a Null value, then it will not be prefixed by a space – even if this property is True. Be careful when using this property with “String” data type fields which are flagged as Never quoted as the space will be considered as part of the field value when the generated file is parsed.
Fixed Width
If the field's “FixedWidth” Meta property is True, then the field always consists of the same number of characters. Unlike Delimited fields, Fixed Width fields are not terminated by a special character. The subsequent field immediately follows the last character of the field. Also, Fixed Length fields are never quoted and do not contain substitutions.
The “Width” field Meta property specifies the number of characters in the field. It must have a value that is one or greater.
If a field's value is Null (file value – not the field's constant value), then raw value written to the file will be a string of Null characters. The Null character is specified by the field's “ValueNullChar” Meta property. The length of this raw value will always be width of the field. Likewise, when reading the value of a FixedWidth field, if all the characters in the raw value are the Null character, then the value is Null.
When reading or writing the raw value for a Fixed Length field it is quite possible that the raw value does not exactly match the width of the field. Several field meta properties determine how to resolve the mismatch between the raw value length and the field width.
Padding Properties (raw value shorter than field width)
If the raw value is shorter than the field width, then the raw value will be padded so that its length becomes equal to the field's width.
The field's “ValuePadAlignment” Meta property indicates whether the padding will occur on the left or the right side of the raw value. This property can also be set to “Auto”, in which case the alignment will automatically be set to the default for each field data type.
The field's “ValuePadCharType” Meta property indicates how the padding character is to be selected. It can have one of the following 3 values:
“Auto”
The padding character is automatically chosen based on the data type of the field. Section X identifies what character is used for each data type. Note that if “Auto” is selected, then the other Padding properties are ignored when reading files – that is, Padding properties are only used when generating files.
“Specified”
The padding character is specified by the “ValuePadChar” field Meta property.on the data type of the field. Note that if “Specified” is selected, then the other Padding properties are ignored when reading files – that is, Padding properties are only used when generating files.
“EndOfValue”
The first padding character after the (unpadded) raw value is the character specified by the “ValueEndOfValueChar” field Meta property. This character flags the end of the value. By selecting “EndOfValue”, the (unpadded) raw value will match the original (unpadded) raw value as the padding can be identified when reading and discarded. Unlike as in the “Auto” and “Specified” values, if “EndOfValue” is selected, then the other Padding properties are used for both reading and writing.
If more than one padding character is required, then all padding characters after the first padding character are ignored when reading. When writing, it is recommended that the “Auto” character be used as the subsequent padding characters.
The above description of “EndOfValue” applies to Right aligned padding. If the padding is left aligned, then the first character before the value will be the character specified by the “ValueEndOfValueChar” field Meta property and all characters prior to this will be ignored when reading.
Truncate Properties (raw value longer than field width)
If the raw value cannot fit into the width of the field, then information will be lost when the file is generated. The field's “ValueTruncateType” Meta property specifies how the truncation is to be done. It can have one of the following values:
“Left”
The left side of the raw value is truncated.
“Right”
The right side of the raw value is truncated.
“TruncateChar”
The raw value is set to a string of repeated characters. The character is specified by the field's “ValueTruncateChar” Meta property. The length of the string matches the width of the field.
“NullChar”
The raw value is set to a string of repeated characters. The character is specified by the field's “ValueNullChar” Meta property. The length of the string matches the width of the field. If this value is selected, then the field value will be considered as Null when it is read.
“Exception”
If a raw value needs to be truncated, the software component or library writing the file should raise an exception. The higher level software can then use this exception to handle this condition.
Note that the Truncate properties are ignored when reading files – they are only applicable when writing files.
Field Data Types
A field can hold different types of data values. The type of data value a field has is specified by the field's “DataType” Meta property. Depending on the type of data a field holds, the field will also have some extra Meta properties which specifies the field's format and style. The format and style properties specify how conversion between the fields value and its raw value is carried out. This is described below in detail for each Field Data type.
Note that a field's value can also be Null – irrespective of its data type. A field's data value is always Null if it is a delimited field which is always quoted and its raw value only contains whitespace. For Fixed Length fields, the data value will be Null if all its characters are the Null Character. Some of the data types also have other conditions in which the field's value is Null. These are also described below.
A field's “DataType” Meta property can have the following values.
“String”
Specifies that the field's data is a string; that is, a series of characters. The string can be up to 2,147,483,647 characters in length.
The Auto Pad character is a space (ASCII 32) and the Auto Pad alignment is Right Pad.
“Boolean”
The field's data is either the boolean True or False. The field Meta properties “FalseText”, “TrueText” and “Styles” specify how to convert between the Field's Value and its Raw Value. This conversion is described in detail in Section X. If the Raw Value is an empty string or only contains only whitespace, then the Value is Null.
The Auto Pad character is a space (ASCII 32) and the Auto Pad alignment is Right Pad.
“Integer”
The field's data is a signed Integer. The range is that of a 64 bit signed integer. The field Meta properties “Format” and “Styles” specify how to convert between the Field's Value and its Raw Value. This conversion is described in detail in Section X. If the Raw Value is an empty string or only contains only whitespace, then the Value is Null.
The Auto Pad character is a space (ASCII 32) and the Auto Pad alignment is Left Pad.
“Float”
The field's data is a Floating point number. The range and precision is that of a IEEE 754 Double Precision (64 bit) Floating Point number . The field Meta properties “Format” and “Styles” specify how to convert between the Field's Value and its Raw Value. This conversion is described in detail in Section X. If the Raw Value is an empty string or only contains only whitespace, then the Value is Null.
The Auto Pad character is a space (ASCII 32) and the Auto Pad alignment is Left Pad.
“Decimal”
The Decimal type is similar to the Float type however it is appropriate for financial calculations requiring large numbers of significant integral and fractional digits and no round-off errors. Decimal numbers support a range of 922337203685477.5808 to 922337203685477.5807 and can have up to 4 decimal places.
Similar to the “Float” data type, the field Meta properties “Format” and “Styles” specify how to convert between the Field's Value and its Raw Value. This conversion is described in detail in Section X. If the Raw Value is an empty string or only contains only whitespace, then the Value is Null.
The Auto Pad character is a space (ASCII 32) and the Auto Pad alignment is Left Pad.
“DateTime”
The field's data is a date and/or time. The date can range from 1 January 0001 to 31 December 9999. The time can be specified down to an accuracy of 10-7 seconds (100 nanosecond). The field Meta properties “Format” and “Styles” specify how to convert between the Field's Value and its Raw Value. This conversion is described in detail in Section X. If the Raw Value is an empty string or only contains only whitespace, then the Value is Null.
The Auto Pad character is a space (ASCII 32) and the Auto Pad alignment is Left Pad.
Heading Lines
Heading Lines are similar to records in that they consist of successive fields of values. However in this case, each value is a heading for the field. They are parsed and generated similarly to records however with the following differences.
Each heading field value is a string – regardless of the data type of the field.
A Field's value becomes one of the headings for that field. The index of the Heading Line determines which heading in the field is set to the value.
A heading line can be blank. If a heading line is blank, all its corresponding heading values in fields are empty strings.
Headings are formatted in the same way as string data type fields. The list below shows the field Meta properties which determine the formatting of heading fields. The equivalent property which affects the formatting of string fields in records is shown in brackets:
“FixedWidth”
“Width”
“HeadingQuotedType” (“ValueQuotedType”)
“HeadingAlwaysWriteOptionalQuote” (“ValueAlwaysWriteOptionalQuote”)
“HeadingWritePrefixSpace” (“ValueWritePrefixSpace”)
“HeadingPadAlignment” (“ValuePadAlignment”)
“HeadingPadCharType” (“ValuePadCharType”)
“HeadingPadChar” (“ValuePadChar”)
“HeadingTruncateType” (“ValueTruncateType”)
“HeadingTruncateChar” (“ValueTruncateChar”)
“HeadingEndOfFieldChar” (“ValueEndOfFieldChar”)
Refer to the field sections for a description of how these properties affect field formatting. The above properties behave in exactly the same way for Heading fields as they for record fields.
The values of Heading Fields are subject to the field's “HeadingConstraint” property.
Heading lines can only have fields as specified by the root sequence or the implicit sequence. Sequence redirections cannot occur in heading lines.
Substitutions
Use of Substitutions
Substitutions allow strings and characters to be represented in a Fielded Text file with tokens. This can be useful in circumstances such as:
Field values contain non-printable characters.
Long strings are used repeatedly within a text file
Field values contain characters which have special purposes within a fielded text file.
A fielded text file is within a context where particular strings should not be present (for example, XML).
Substitutions are analogous to escape sequences as used in the C programming language though not exactly the same.
Identifying Substitutions
Substitutions exist within Field Text as pairs of characters. The first character of the pair is the Substitution Character. This character is analogous to the Escape Character in 'C' escape sequences. The Meta property “SubstitutionChar” specifies which character is the substitution character for the file.
The second character is the token character. If a substitution is defined in the Meta which specifies this token, then this substitution will be applied at this point in the file. If no substitution has this token, then the token character itself is the substitution.
Note that substitutions are only enabled if the “SubstitutionsEnabled” Meta property is True. If this property is not True, then the Meta “SubstitutionChar” property is ignore and all substitutions defined in the Meta are also ignored. Also, substitutions do not occur in Fixed Width fields.
Substitution Elements
The substitutions are specified by “Substitution” elements in the Meta. There is one element for each substitution. A Substitution element has the following Meta properties:
“Token”
Specifies the character which is the token for this substitution.
“Type”
There are several different types of substitutions. This property can have one of the following values:
“String”
The substitution pair (Substitution character and token) are replaced by a string. The string is specified by the Substitution's “Value” Meta property
“AutoEndOfLine”
The substitution pair are replaced by the EoL character(s) specified by the “ EndOfLineAutoWriteType” main Meta property. Note that if the “ EndOfLineAutoWriteType” has a value of “Local”, then these substitutions will place EoL character(s) into the text that are easily readable on the local computer.
“XmlCharLT”, “XmlCharGT”, “XmlCharAmp”, “XmlCharApos”, “XmlCharQuot”
These 5 types will respectively replace a substitution pair with one of the following 5 characters: <, >, &, ', ". These characters are the 5 XML built-in character entities which cannot be included inside an XML element. If these substitution elements are included, then their respective characters will not be included in field text and it may be easier to include the text within XML elements.
“Value”
Specifies the string or character to be substituted if the substitution type is “String”.
Processing Substitutions
Substitutions are processed when converting raw values to field text (writing) and when converting field text to raw values (reading).
When reading, if a string gets substituted into a field, then none of the characters in a substituted string (or the single character) have any special significance – they are simply treated as characters in the raw value. For example, if a character in a substituted string is the same as the character specified by the Meta “DelimiterChar” property, then that character is not treated as a delimiter. This also applies to characters which follow the substitution character but are not included as a token in any substitution elements.
When writing, the raw value is scanned for strings or characters which can be substituted. Where-ever possible, strings or characters in the raw value which match one of the substitution elements, should be replaced by that substitution. If at any point in the processing, more than one substitution can be applied, then the one declared earlier in the Meta will have priority and be used.
Sequences
Sequences define the order in which fields occur in records. Each sequence has an ordered list of sequence items. Each sequence item specifies:
A Field
Zero or more Sequence Redirects
The order of fields correspond to the fields in the ordered list of sequence items. However after each field's value is read or written, the value is compared to the conditions specified by the redirects. If the value matches any condition, then the redirect specifies a new sequence. This new sequence can either be invoked immediately after the current field, or after the end of the current sequence. When it is invoked, it specifies the order of fields from that point onwards in the record.
Sequence Elements
Sequences are specified by “Sequence” elements in the Meta.
Each element is identified by the Sequence's “Name” Meta property. All Sequences need to have a unique name. However sequence names are not case sensitive. So if 2 sequences have names that only differ by the casing of letters, then these names are not unique.
Only one sequence can have the sequence Meta property “Root” with a value of “True”. This defines that sequence as the Root sequence. Whenever a record (or heading line) is parsed, the root sequence is used to determine the order of the fields from the 1st field onwards. Note that for records (but not heading lines) other sequences can be invoked during the parsing of a records fields, if any of the field values match a redirect condition.
If none of the sequences have a “Root” property set to True, then the root sequence will be the first declared in the Meta which does not have a “Root” Meta property.
A Sequence's items (SequenceItems) can be defined in 2 ways:
Sequence Item elements
The “SequenceItem” element is used to define the items. The implicit order of these items is the order they appear in the Meta. However a Sequence Item can explicitly specify its position in the order with its “Index” property.
FieldIndices property
Sequence Elements can also be specified by the FieldIndices property. This property contains an array of Integers in Standardised Integer CommaText format. Each integer is the index of the a field (in the resolved fields order).
The array specifies a list of Sequence Items, each of which has the field specified by the Integer value. These Sequence Items cannot have an “Index” property so their position is always specified by the implicit order. They also do not have any redirects.
The resolved order of the items is determined by the implicit order of those specified by elements, followed by those specified by the “FieldIndices” property. Items specified by an element, however can have their position explicitly set. This is further explained in section X.
If the Meta does not contain any sequences, then the order of fields is determined from the “implicit sequence”. The implicit sequence is determined from the resolved order of fields as described in Section X.
Sequence Item elements
A Sequence can have zero or more Sequence Item elements.
Each Sequence Item must specify an existing Field. If a Sequence Item was specified with a “SequenceItem” element, then the element must contain the “FieldIndex” property. This property specifies the index of a field using the fields' resolved order.
If an Item was specified with its Sequence's “FieldIndices” property, then its (resolved) field index will be its Integer value in the “FieldIndices” integer array.
Sequence Item's specified by an element can also have an “Index” property. The value of this property explicitly specifies the Items position in its Sequence's resolved order of Sequence Items.
Sequence Items can contain zero or more Sequence Redirects. A Sequence Redirect is specified by a “Redirect” element within the Sequence Item element.
Sequence Redirect elements
A Sequence Redirect is used to change the order of fields in a record. A Redirect specifies a condition, a sequence and an invokation delay. If the field's value matches the condition, then the fields in the sequence specified in the redirect will be used in the record. The Redirect can specify that the new fields should be invoked immediately after the current field or they can be invoked after the current sequence is finished.
A record can have more than one redirect invoke a new sequence.
The Redirect Meta property “SequenceName” specifies the name of the sequence a redirect will invoke.
The Redirect Meta property “InvokationDelay” specifies whether the fields in the new sequence are to be invoked immediately after the current field or after the current sequence is finished. If a Redirect specifies that new fields should be invoked after the current sequence ends, then any Redirects in subsequent Sequence Items in the current sequence are to be ignored.
A sequence item can have more than one redirect. When checking for a match, the first redirect whose condition matches is applied. The redirects have a resolved order. This order is determined by the order in which they appear in the Meta (the implicit order) and whether a redirect has an “Index” Meta property. The “Index” Meta property explicitly specifies the order position for that sequence redirect. Section X details how the resolved order is determined from implicit element order and the “Index” property.
The “Type” Redirect Meta property specifies what type of condition a Meta contains. It can have one of the following values:
“ExactString”
The field value needs to match the string stored in the Redirect's “Value” Meta property. The comparison is case sensitive.
“CaseInsensitiveString”
The field value needs to match the string stored in the Redirect's “Value” Meta property. The comparison is not case sensitive and uses the culture specified by the Meta “Culture” property.
“Boolean”
The field value needs to match the Boolean value stored in the Redirect's “Value” Meta property. The Boolean is stored in the Meta using Standardised Boolean formatting.
“ExactInteger”
The field value needs to exactly match the Integer value stored in the Redirect's “Value” Meta property. The Integer is stored in the Meta using Standardised Integer formatting.
“ExactFloat”
The field value needs to exactly match the Float value stored in the Redirect's “Value” Meta property. The Float is stored in the Meta using Standardised Float formatting.
“ExactDateTime”
The field value needs to exactly match the DateTime value stored in the Redirect's “Value” Meta property. The DateTime is stored in the Meta using Standardised DateTime formatting.
“Date”
The date portion of the field value needs to exactly match the date portion of the DateTime value stored in the Redirect's “Value” Meta property. The Time portions are ignored in the comparison. The DateTime is stored in the Meta using Standardised DateTime formatting.
“ExactDecimal”
The field value needs to exactly match the Decimal value stored in the Redirect's “Value” Meta property. The Decimal is stored in the Meta using Standardised Float formatting.
If a Sequence Item contains a Redirect condition whose data type differs from the Sequence Item's field's data type, then the comparison should be undertaken with the following steps:
Try to convert the field's value to the same data type as expected by the Redirect condition. If this conversion fails, then the field value is considered as not matching the condition.
Carry out the comparison with the converted field value.
It is recommended that Sequence Items use Redirects which expect the same data type as provided by the Sequence Item's field. Different implementations of Fielded Text may carry out Data Type conversions in different manners – leading to different interpretations of the data in a fielded text file.
Design Only Meta
Calculating resolved order of elements
Value Formats and Styles
Standardised Formatting
Standardised Formatting allows you to express values as strings using a fixed format which is not affected by culture. The Invariant culture is always used when comparing text or determining characters of special significance.
In Standardised Formatting only the following characters are considered as Whitespace characters:
Horizontal Tab (ASCII 9)
Line Feed (ASCII 10)
Vertical Tab (ASCII 11)
Form Feed (ASCII 12)
Carriage Return (ASCII 13)
Space (ASCII 32)
The following Data Types and Data Type arrays have standardised formats:
Boolean
Standardised Booleans can be one of the following values:
“True”
“False”
These values are not case sensitive and are compared with the invariant culture.
Integer
Standardised Integers have the following format:
[ws][sign]digits[ws]
Items in square brackets ([ and ]) are optional; and the values of the other items are as follows.
ws
An optional white space.
sign
An optional sign.
digits
A sequence of digits ranging from 0 to 9.
Float
Both Float and Decimal numbers can be stored in the Standardised Float Format.
Standardised Float numbers have the following format:
[ws][sign]integral-digits[.[fractional-digits]][e[sign]exponential-digits][ws]
Optional items are framed in square brackets ([ and ]). Items containing the term "digits" consist of a series of numeric characters ranging from 0 to 9. The values of the items are:
ws
A series of white space characters.
sign
A negative sign or positive sign symbol.
integral-digits
A series of digits specifying the integral part of the number. Integral-digits can be absent if there are fractional-digits.
'.'
A culture invariant decimal point symbol (a period).
fractional-digits
A series of digits specifying the fractional part of the number.
'e'
An uppercase or lowercase character 'e', indicating exponential (scientific) notation.
exponential-digits
A series of digits specifying an exponent.
DateTime
Standardised DateTimes have one of the following formats:
yyyyMMdd
Specifies a DateTime where the time portion is either 0 or irrelevant. It consists of an 8 character string containing a 4 digit year, a 2 digit month and a 2 digit day.
hhmmss
Specifies a DateTime where the date portion is irrelevant. It consists of 6 character string containing a 2 digit hour, a 2 digit minute and a 2 digit seconds.
yyyyMMddhhmmss
Specifies a DateTime with both date and time portions. The first 8 characters specify the date as described above. The next 6 characters specify the time as described above.
yyyyMMddhhmmss.<digit>[0 to 6 <digits>]
Specifies a DateTime with fractional seconds. The first 8 characters specify the date as described above. The next 6 characters specify the time as described above. This is then followed by a period character (ASCII 46) and at least one digit (ranging from 0 to9). The digits after the period specify the fractional seconds. Up to 7 digits can be used to specify the fractional seconds – allowing time to be specified to an accuracy of 100 nanoseconds.
'hhmmss.<digit>[0 to 6 <digits>]
Specifies a Time with fractional seconds. The date portion of the DateTime is irrelevant. The first 6 characters specify the time as described above. This is then followed by a period character (ASCII 46) and at least one digit (ranging from 0 to9). The digits after the period specify the fractional seconds. Up to 7 digits can be used to specify the fractional seconds – allowing time to be specified to an accuracy of 100 nanoseconds.
String Quoting
Standardised String Quoting is implemented as follows:
The Quote character is always " (ASCII 34).
All existing Quote Characters in a string are replaced with 2 Quote Characters ("").
A Quote Character is placed at the start and the end of the string.
String CommaText
Standardised String CommaText allows you to express an array of strings as a single string. It is implemented as follows:
A string in the array must be quoted with standardised quoting if it meets any of the following conditions.
The string contains one or more Comma Characters , (ASCII 44)
The first non-white space character in the string is the Quote Character (ASCII 34).
Even if a string in the array does not meet any of the above conditions, it still can be optionally quoted with standardised string quoting.
The resultant strings (after quoting if required) are then combined into one long string with separators. This is done by placing them in the string in their order according the array. The separator between each array string is one Comma character (ASCII 44).
If an array string is quoted, then optional white space characters can be placed between the enclosing quote characters and the Comma Character (or the start or end of the resultant string if the array string is respectively the first or last in the array). Otherwise, no other characters can appear in the string.
Integer CommaText
Standardised Integer CommaText allows you to express an array of Integers as a single string. It is implemented as follows:
Each Integer in the array is converted to a string using Standardised Integer formatting.
The resultant string array is then converted to a single string using Standardised String CommaText formatting.
Custom Formatting
Boolean
Basic Read Format
Two Meta properties determine whether a text string represents a True or False value:
The “FalseText” Meta property specifies the string which represents a value of False.
The “TrueText” Meta property specifies the string which represents a value of True.
When a text string is parsed, it is firstly compared against the Meta TrueText string. If these 2 strings match, then the value is True. If they do not match, then the text string is compared with the Meta FalseText string. If these match then the Value is False, otherwise there is a parsing error.
Read Styles
Read Styles provide some additional capabilities when parsing a text string. These styles only apply to parsing and not when generating strings.
IgnoreCase
Case is ignored when text strings are compared to the Meta FalseText and TrueText strings. The selected culture is used when doing case insensitive comparisons.
MatchFirstCharOnly
Only the first characters of the text string and the Meta FalseText and TrueText strings are compared.
IgnoreTrailingChars
If the text string is longer than the Meta FalseText, then the comparison ignores all characters in the text string which are beyond the length of the Meta FalseText string. For example, if this style is selected then the text string “Failure” would match the Meta FalseText “Fail”. The same applies when comparing the text string with the Meta TrueText.
FalseIfNotMatchTrue
If the text string matches the Meta TrueText, then the value is True. If it does not match, then the value is False. That is, the text string is not compared against the Meta FalseText.
Write Format
Either the string specified by the FalseText or TrueText Meta property is written to a file depending on the value of the boolean.
Integer
Basic Read Format
When parsing Integer strings, the basic format of a string is as follows.
[ws][sign]digits[ws]
Items in square brackets ([ and ]) are optional; and the values of the other items are as follows.
ws
An optional white space.
sign
An optional sign.
digits
A sequence of digits ranging from 0 to 9.
Read Styles
Read Styles either restrict or allow additional formatting on strings. The styles affect the parsing of fields but not the generating of fields.
Integers can have the Numeric styles applied as described in Section X. Note that not all numeric styles are applicable to Integers.
The styles themselves are stored in the “Styles” Meta Property. This property contains the names of all applied styles as a Standardised Comma Text string.
Write Format
Culture Dependencies
Float
Read Format
When parsing floating point number strings, the basic format of a string is as follows.
[ws][sign]integral-digits[.[fractional-digits]][e[sign]exponential-digits][ws]
Optional items are framed in square brackets ([ and ]). Items containing the term "digits" consist of a series of numeric characters ranging from 0 to 9.
ws
A series of white space characters.
Sign
A negative sign or positive sign symbol.
integral-digits
A series of digits specifying the integral part of the number. Runs of integral-digits can be partitioned by a group-separator symbol. (For example, in some cultures a comma (,) separates groups of thousands.) Integral-digits can be absent if there are fractional-digits.
'.'
A culture-specific decimal point symbol.
fractional-digits
A series of digits specifying the fractional part of the number.
'e'
An uppercase or lowercase character 'e', indicating exponential (scientific) notation.
exponential-digits
A series of digits specifying an exponent.
Read Styles
Read Styles either restrict or allow additional formatting on strings. The styles affect the parsing of fields but not the generating of fields.
Floating point numbers can have the Numeric styles applied as described in Section X. Note that not all numeric styles are applicable to floating point numbers.
The styles themselves are stored in the “Styles” Meta Property. This property contains the names of all applied styles as a Standardised Comma Text string.
Write Formatting
Culture Dependencies
Numeric Read Styles
Numeric Styles either restrict or allow additional formatting on strings. They are used when strings containing integers or floating point numbers are parsed.
There are 2 types of Numeric Read Styles: Basic and Composite.
A Basic Style specifies a particular restriction or enhancement to formatting.
A Composite Style specifies a set of Basic styles. Composite styles are provided to conveniently specify combinations of styles frequently used.
Basic Styles
AllowCurrencySymbol
The string is parsed as a currency if it contains the currency symbol. The selected culture determines which character is the currency symbol and how currencies should be parsed.
AllowDecimalPoint
The string can contain a decimal point. The selected culture determines which character is used as the decimal point.
AllowExponent
The string can be in exponential notation.
AllowHexSpecifier
The string contains a hexadecimal string. In addition to digits (0-9), strings can also contain characters A-F and a-f.
AllowLeadingSign
The number in the string can have a leading sign. The selected culture determines which character is used as the positive sign and the negative sign.
AllowLeadingWhite
Any leading whitespace characters in the string are ignored when parsing.
AllowParentheses
The number in the string can be enclosed in one pair of parentheses.
AllowThousands
The number in the string can be separated into groups, for example a character can be placed between every 3 digits to show thousands. The selected culture determines how digits are grouped.
AllowTrailingSign
The number in the string can have a trailing sign. The selected culture determines which character is used as the positive sign and the negative sign.
AllowTrailingWhite
Any trailing whitespace characters in the string are ignored when parsing.
Composite Styles
Any
All styles are used except AllowHexSpecifier
Currency
All styles are used except AllowHexSpecifier and AllowExponent
Float
AllowLeadingWhite, AllowTrailingWhite, AllowDecimalPoint and AllowExponent styles are used.
HexNumber
AllowLeadingWhite, AllowTrailingWhite and AllowHexSpecifier styles are used.
Integer
AllowLeadingWhite, AllowTrailingWhite and AllowLeadingSign styles are used.
None
No styles are used.
Number
AllowLeadingWhite, AllowTrailingWhite, AllowLeadingSign, AllowTrailingSign, AllowDecimalPoint and AllowThousands styles are used.
Numeric Write Formatting
DateTime
Read Format
The format of the datetime in a string is specified by the “Format” Meta property. The “Format” property contains a string which specifies how a DateTime is formated as text. It is described in section X below.
Read Styles
Read Styles provide some additional capabilities when parsing a text string. These styles only apply to parsing and not when generating strings.
AllowInnerWhite
Extra whitespace characters in the text string are ignored when parsing – unless the whitespace characters are specified in the format string.
Write Formatting
DateTime values are converted to text strings using the format specified by the “Format” Meta property.
DateTime Format string
Invariant Culture
The invariant culture is used when cultural conventions want to be avoided as far as possible. This culture is associated with the English language and has the following settings:
Numeric
Currency Symbol: “$”
Currency Decimal Separator: “.”
Number Decimal Separator: “.”
Negative Sign: “-”
Positive Sign: “+”
Currency Group Separator: “,”
Number Group Separator: “,”
Currency Group Size: 3
Number Group Size: 3
DateTime
Meta Reference
Root
FieldedText
Main/Root
General Attributes
Culture
CharEncoding
AutoDetectCharEncoding
EndOfLineType
EndOfLineChar
EndOfLineAutoWriteType
QuoteChar
DelimiterChar
LineCommentChar
StuffedEmbeddedQuotes
SubstitutionEnabled
SubstitutionChar
AllowEndOfLineCharInQuotes
IgnoreBlankLines
IgnoreExtraChars
HeadingLineCount
MainHeadingLineIndex
HeadingConstraint
HeadingQuotedType
HeadingAlwaysWriteOptionalQuote
HeadingWritePrefixSpace
HeadingPadAlignment
HeadingPadCharType
HeadingPadChar
HeadingTruncateType
HeadingTruncateChar
HeadingEndOfValueChar
Design Only Attributes
NewBooleanFieldFalseText
NewBooleanFieldTrueText
NewIntegerFieldFormat
NewFloatFieldFormat
NewDateTimeFieldFormat
NewDecimalFieldFormat
NewBooleanFieldStyles
NewIntegerFieldStyles
NewFloatFieldStyles
NewDateTimeFieldStyles
NewDecimalFieldStyles
Elements
Field
Substitution
Sequence
Field
Attributes
DataType
Index
Id
Name
FixedWidth
Width
HeadingConstraint
Constant
ValueQuotedType
ValueAlwaysWriteOptionalQuote
ValueWritePrefixSpace
ValuePadAlignment
ValuePadCharType
ValuePadChar
ValueTruncateType
ValueTruncateChar
ValueEndOfValueChar
ValueNullChar
HeadingQuotedType
HeadingAlwaysWriteOptionalQuote
HeadingWritePrefixSpace
HeadingPadAlignment
HeadingPadCharType
HeadingPadChar
HeadingTruncateType
HeadingTruncateChar
HeadingEndOfValueChar
Headings
Null
Value
Format
Styles
FalseText
TrueText
Elements
Substitution
Attributes
Type
Token
Value
Elements
Sequence
Attributes
Name
Root
FieldIndices
Elements
Item
SequenceItem
Attributes
Index
FieldIndex
Elements
Redirect
Redirects
Attributes
Index
Type
SequenceName
InvokationDelay
Elements
Other standards and conventions
Following are some recommendations on how other standards and conventions can incorporate Fielded Text files
Mime Type
The Mime Type “text/fielded” be reserved for Fielded Text content
File Name Extensions
The extension “ftx” be used in names for files which contain Declared Fielded Text content.
The extension “ftm” be used in names for files which contain Fielded Text Meta data
Acknowledgements
.NET 1.1
Standardised Integers
Standardised Floats
Custom Integer
Custom Float
Custom Numeric Read Styles
Custom Numeric Write Formatting
Custom DateTime
Appendices
Appreviations
EoL End Of Line
Page