byte_source Class Reference

base that does all the work except getting bytes from stream More...

#include <storexml.h>

Inheritance diagram for byte_source:

List of all members.

Public Member Functions

byte_source (mem_pool_t &small_pool, mem_pool_t &big_pool, size_t xml_size, const char *url, bool subset)

constructor

virtual ~byte_source ()

destructor

xml_forceinline ub1_t pick ()

picks byte

xml_forceinline ub1_t pop ()

pops byte operator++(int)

xml_forceinline ub1_t pip ()

pips byte operator++()

xml_forceinline void skip_white_space (bool mustPresent=false, const char *message=0)

skips junk chars defined by is_white_space() function if not then throw exception with specified message

void throw_exception (const char *msg_text)

throws exception adding line and char postion information

void skip_string (const char *x, const char *message)

skips string if not, then throws exception with specified message

void skip_sign (ub1_t symbol, bool skip_before, bool skip_after, const char *message)

skips one byte if not, then throws exception with specified message

ub1_t skip_quote (ub1_t symbol=0)

skips quote symbol it can be single quote or double quote if input parameter = 0 then function detects the type of quote and returns the detected quote symbol if input parameter = quote symbol function skip the specified quote symbol in case of problems function throws exception

size_t current_pos () const

returns current char position

void push (const ub1_t *x, size_t len)

inserts into head of sequence

void push (ub1_t x)

pushes byte back to buffer head

size_t pull (ub1_t *x, size_t len)

pulls block of bytes from stream

encodingSchema get_encoding () const

returns detected encoding schema

void set_encoding (encodingSchema schema)

sets encoding

const char * get_location () const

gets stream location

size_t get_standalone () const

gets standalone flag

bool get_subset () const

gets subset flag

size_t get_version () const

gets version

size_t get_xml_size () const

xml size

Protected Member Functions

virtual bool data_request (ub1_t *buf, size_t &len)=0

pure virtual function getting bytes from stream

encodingSchema parseXMLDecl ()

parses xml declaration [23] XMLDecl ::= '<?xml' VersionInfo EncodingDecl? SDDecl? S? '?>'

encodingSchema parseTextDecl ()

parses text declaration [77] TextDecl ::= '<?xml' VersionInfo? EncodingDecl S? '?>'

size_t get_buffer_pos () const

gets buffer position

void reset_buffer ()

resets internal buffer

Protected Attributes

byte_allocator * _depot_allocator

depot allocator

byte_allocator * _convert_allocator

conversion allocator

byte_allocator * _store_allocator

temporary store allocator

byte_allocator * _list_allocator

store buffers allocator

mem_pool_t & _small_pool

small memory pool

mem_pool_t & _big_pool

big memory pool

const size_t _xml_size

xml size

string_t _url

the location of xml stream

Private Member Functions

bool convert_chars (size_t count)

converts count bytes from stream to internal buffer

bool taste_buffer ()

checks the encoding

bool auto_convert ()

does auto conversion

ub1_t go_shopping ()

gets bytes from stream

bool parseXMLDeclInfo ()

parses xml declaration info

void parseVersion ()

parses version

encodingSchema parseEncoding ()

parses encoding

size_t parseStandalone ()

parses standalone info

encodingSchema checkEncodingSchema (const char *schema)

validates the specified schema

void scanQuotedValue (ub1_t *value, size_t len, bool(*fn)(ub1_t), const char *message)

scans the quoted value to make sure it is valid

Private Attributes

bool _subset

subset flag

ub1_t * _buffer

current buffer

ub1_t * _convert_buffer

convert buffer

size_t _buffer_pos

current buffer position

ub1_t _symbol

current symbol

store_list_t _active_store

store buffers in use

store_list_t _used_store

store buffers for reusing

size_t _line_counter

line number

size_t _char_counter

char position (for multibyte chars)

size_t _pos_counter

byte position

encodingSchema _encodingSchema

encoding schema

size_t _version

version

size_t _standalone

standalone flag

bool _end

end of stream flag

Detailed Description

base that does all the work except getting bytes from stream

Definition at line 45 of file storexml.h.

Constructor & Destructor Documentation

byte_source::byte_source	(	mem_pool_t &	small_pool,
		mem_pool_t &	big_pool,
		size_t	xml_size,
		const char *	url,
		bool	subset
	)

constructor

Parameters:

small_pool	small memory pool
big_pool	big memory pool
xml_size	xml size - just a tip
url	optional location
subset	subset flag

Definition at line 46 of file storexml.cpp.

References _big_pool, _buffer, _convert_allocator, _convert_buffer, _depot_allocator, _list_allocator, _small_pool, _store_allocator, _xml_size, byte_allocator::allocate(), pool< C >::loan_object(), and os_def_size.

byte_source::~byte_source ( ) [virtual]

destructor

Definition at line 80 of file storexml.cpp.

References _big_pool, _convert_allocator, _depot_allocator, _list_allocator, _small_pool, _store_allocator, _xml_size, os_def_size, and pool< C >::return_object().

Member Function Documentation

xml_forceinline ub1_t byte_source::pick ( )

picks byte

Definition at line 67 of file storexml.hpp.

References _buffer_pos, _symbol, _xml_size, and go_shopping().

Referenced by parseTextDecl(), parseXMLDecl(), byte_manager::pick(), pip(), scanQuotedValue(), skip_quote(), skip_string(), and skip_white_space().

xml_forceinline ub1_t byte_source::pop ( )

pops byte operator++(int)

Definition at line 75 of file storexml.hpp.

References _buffer, _buffer_pos, _char_counter, _line_counter, _pos_counter, _symbol, _xml_size, ch_cr, ch_lf, and go_shopping().

Referenced by parseTextDecl(), parseXMLDecl(), pip(), byte_manager::pop(), scanQuotedValue(), skip_quote(), skip_string(), and skip_white_space().

xml_forceinline ub1_t byte_source::pip ( )

pips byte operator++()

Definition at line 94 of file storexml.hpp.

References pick(), and pop().

Referenced by byte_manager::pip(), and skip_sign().

xml_forceinline void byte_source::skip_white_space	(	bool	mustPresent = `false`,
		const char *	message = `0`
	)

skips junk chars defined by is_white_space() function if not then throw exception with specified message

Parameters:

mustPresent	flag of white space presence
message	error message

Definition at line 103 of file storexml.hpp.

References ch_cr, ch_hor_tab, ch_lf, ch_space, pick(), pop(), and throw_exception().

Referenced by parseTextDecl(), parseXMLDecl(), skip_sign(), and byte_manager::skip_white_space().

void byte_source::throw_exception ( const char * msg_text )

throws exception adding line and char postion information

Parameters:

msg_text

message text

Definition at line 116 of file storexml.cpp.

References _char_counter, _line_counter, exception::_throw(), base_string< T >::append(), str_template::strlen(), and str_template::strprint().

Referenced by convert_chars(), parseEncoding(), byte_manager::parseName(), parseStandalone(), parseTextDecl(), parseVersion(), parseXMLDecl(), parseXMLDeclInfo(), push(), scanQuotedValue(), skip_quote(), skip_sign(), skip_string(), skip_white_space(), taste_buffer(), and byte_manager::throw_exception().

void byte_source::skip_string	(	const char *	x,
		const char *	message
	)

skips string if not, then throws exception with specified message

Parameters:

x	string to skip
message	error message

Definition at line 926 of file storexml.cpp.

References pick(), pop(), and throw_exception().

Referenced by parseEncoding(), parseStandalone(), parseVersion(), and byte_manager::skip_string().

void byte_source::skip_sign	(	ub1_t	symbol,
		bool	skip_before,
		bool	skip_after,
		const char *	message
	)

skips one byte if not, then throws exception with specified message

Parameters:

symbol	byte to skip
skip_before	check white spaces before
skip_after	check white spaces after
message	error message

Definition at line 936 of file storexml.cpp.

References pip(), skip_white_space(), and throw_exception().

Referenced by parseEncoding(), parseStandalone(), parseVersion(), and byte_manager::skip_sign().

ub1_t byte_source::skip_quote ( ub1_t symbol = 0 )

skips quote symbol it can be single quote or double quote if input parameter = 0 then function detects the type of quote and returns the detected quote symbol if input parameter = quote symbol function skip the specified quote symbol in case of problems function throws exception

Parameters:

symbol

optional quote symbol

Definition at line 979 of file storexml.cpp.

References ch_double_quote, ch_single_quote, pick(), pop(), and throw_exception().

Referenced by byte_manager::parseQuotedValue(), scanQuotedValue(), and byte_manager::skip_quote().

xml_forceinline size_t byte_source::current_pos ( ) const [inline]

returns current char position

Definition at line 129 of file storexml.hpp.

References _pos_counter.

Referenced by byte_manager::resolveEntity().

void byte_source::push	(	const ub1_t *	x,
		size_t	len
	)

inserts into head of sequence

Parameters:

x	buffer
len	buffer length

Definition at line 134 of file storexml.cpp.

References _active_store, _buffer, _buffer_pos, _list_allocator, _pos_counter, _store_allocator, _symbol, _used_store, _xml_size, byte_allocator::allocate(), base_list< T >::empty(), base_list< T >::front(), _list< T, A >::pop_front(), _list< T, A >::push_front(), and throw_exception().

Referenced by push(), and byte_manager::push().

xml_forceinline void byte_source::push ( ub1_t x ) [inline]

pushes byte back to buffer head

Parameters:

x

byte

Definition at line 136 of file storexml.hpp.

References push().

size_t byte_source::pull	(	ub1_t *	x,
		size_t	len
	)

pulls block of bytes from stream

Parameters:

x	pre-allocated buffer
len	buffer length

Definition at line 176 of file storexml.cpp.

References _buffer, _buffer_pos, _char_counter, _end, _pos_counter, _xml_size, and go_shopping().

Referenced by buffer_loader::load().

encodingSchema byte_source::get_encoding ( ) const [inline]

returns detected encoding schema

Definition at line 143 of file storexml.hpp.

References _encodingSchema.

Referenced by parseXMLDeclInfo().

void byte_source::set_encoding ( encodingSchema schema ) [inline]

sets encoding

Definition at line 150 of file storexml.hpp.

References _encodingSchema.

Referenced by parseXMLDeclInfo().

const char * byte_source::get_location ( ) const [inline]

gets stream location

Definition at line 157 of file storexml.hpp.

References _url.

Referenced by dtd_processor::expandPEReference(), xml_processor::parseDTD(), and byte_manager::resolveEntity().

virtual bool byte_source::data_request	(	ub1_t *	buf,
		size_t &	len
	)			`[protected, pure virtual]`

pure virtual function getting bytes from stream

Parameters:

buf	pre-allocated buffer
len	[in,out] [in] buffer length, [out] retrieved bytes

Implemented in stream_input_memory, and stream_input_common.

Referenced by convert_chars(), and taste_buffer().

encodingSchema byte_source::parseXMLDecl ( ) [protected]

parses xml declaration [23] XMLDecl ::= '<?xml' VersionInfo EncodingDecl? SDDecl? S? '?>'

Definition at line 841 of file storexml.cpp.

References _encodingSchema, _standalone, _version, AUTO, ch_close_angle, ch_e, ch_question, ch_s, ch_v, parseEncoding(), parseStandalone(), parseVersion(), pick(), pop(), skip_white_space(), and throw_exception().

Referenced by parseXMLDeclInfo().

encodingSchema byte_source::parseTextDecl ( ) [protected]

parses text declaration [77] TextDecl ::= '<?xml' VersionInfo? EncodingDecl S? '?>'

Definition at line 891 of file storexml.cpp.

References _encodingSchema, _version, ch_close_angle, ch_e, ch_question, ch_v, parseEncoding(), parseVersion(), pick(), pop(), skip_white_space(), and throw_exception().

Referenced by parseXMLDeclInfo().

bool byte_source::convert_chars ( size_t count ) [private]

converts count bytes from stream to internal buffer

Definition at line 532 of file storexml.cpp.

References _buffer, _buffer_pos, _convert_buffer, _encodingSchema, _symbol, _xml_size, base_string< T >::append(), data_request(), fixedN_to_utf8(), isoN_to_utf8(), throw_exception(), UCS_4B, UCS_4BS, UCS_4L, UCS_4LS, US_ASCII, utf8_to_utf8(), UTF_16B, UTF_16L, UTF_8, UTF_ISO88591, WINDOWS_1251, and windowsN_to_utf8().

Referenced by auto_convert(), parseXMLDeclInfo(), and taste_buffer().

bool byte_source::taste_buffer ( ) [private]

checks the encoding

Definition at line 251 of file storexml.cpp.

References _buffer, _buffer_pos, _convert_buffer, _encodingSchema, _symbol, convert_chars(), data_request(), fixedN_to_utf8(), parseXMLDeclInfo(), throw_exception(), UCS4BBOM, UCS4BPre, UCS4LBOM, UCS4LPre, UCS_4B, UCS_4L, UTF16BBOM, UTF16BPre, UTF16LBOM, UTF16LPre, utf8_to_utf8(), UTF8BOM, UTF8Pre, UTF_16B, UTF_16L, and UTF_8.

Referenced by go_shopping().

bool byte_source::auto_convert ( ) [private]

does auto conversion

Definition at line 525 of file storexml.cpp.

References _buffer_pos, _xml_size, and convert_chars().

Referenced by go_shopping(), and parseXMLDeclInfo().

ub1_t byte_source::go_shopping ( ) [private]

gets bytes from stream

Definition at line 202 of file storexml.cpp.

References _active_store, _buffer, _buffer_pos, _encodingSchema, _end, _list_allocator, _symbol, _used_store, _xml_size, AUTO, auto_convert(), base_list< T >::empty(), base_list< T >::front(), _list< T, A >::pop_front(), _list< T, A >::push_front(), and taste_buffer().

Referenced by pick(), pop(), and pull().

bool byte_source::parseXMLDeclInfo ( ) [private]

parses xml declaration info

Definition at line 468 of file storexml.cpp.

References _big_pool, _buffer, _buffer_pos, _encodingSchema, _small_pool, _standalone, _subset, _version, _xml_size, AUTO, auto_convert(), ch_close_angle, ch_question, convert_chars(), get_encoding(), get_standalone(), get_version(), parseTextDecl(), parseXMLDecl(), set_encoding(), throw_exception(), UCS_4, UCS_4B, UCS_4BS, UCS_4L, UCS_4LS, UTF_16, UTF_16B, UTF_16L, and UTF_8.

Referenced by taste_buffer().

void byte_source::parseVersion ( ) [private]

parses version

Definition at line 782 of file storexml.cpp.

References _convert_buffer, _version, ch_equal, is_versionnum_char(), os_minus_one, scanQuotedValue(), skip_sign(), skip_string(), str_version, str_template::strnocasecmp(), throw_exception(), and xml_decl_max_len.

Referenced by parseTextDecl(), and parseXMLDecl().

encodingSchema byte_source::parseEncoding ( ) [private]

parses encoding

Definition at line 800 of file storexml.cpp.

References _convert_buffer, ch_equal, checkEncodingSchema(), is_encname_char(), is_letter(), scanQuotedValue(), skip_sign(), skip_string(), str_encoding, throw_exception(), and xml_decl_max_len.

Referenced by parseTextDecl(), and parseXMLDecl().

size_t byte_source::parseStandalone ( ) [private]

parses standalone info

Definition at line 820 of file storexml.cpp.

References _convert_buffer, ch_equal, os_minus_one, scanQuotedValue(), skip_sign(), skip_string(), str_no, str_standalone, str_yes, str_template::strcmp(), throw_exception(), and xml_decl_max_len.

Referenced by parseXMLDecl().

encodingSchema byte_source::checkEncodingSchema ( const char * schema ) [private]

validates the specified schema

Parameters:

schema

external encoding

Definition at line 949 of file storexml.cpp.

References AUTO, os_minus_one, str_template::strnocasecmp(), UCS_4, UCS_4B, UCS_4L, US_ASCII, UTF_16, UTF_16B, UTF_16L, UTF_8, UTF_ISO88591, and WINDOWS_1251.

Referenced by parseEncoding().

void byte_source::scanQuotedValue	(	ub1_t *	value,
		size_t	len,
		bool(*)(ub1_t)	fn,
		const char *	message
	)			`[private]`

scans the quoted value to make sure it is valid

Parameters:

value	pre-allocated buffer
len	buffer length
fn	validity function
message	error message

Definition at line 996 of file storexml.cpp.

References pick(), pop(), skip_quote(), and throw_exception().

Referenced by parseEncoding(), parseStandalone(), and parseVersion().

BEGIN_TERIMBER_NAMESPACE size_t byte_source::get_standalone ( ) const [inline]

gets standalone flag

Definition at line 39 of file storexml.hpp.

References _standalone.

Referenced by byte_manager::get_standalone(), and parseXMLDeclInfo().

bool byte_source::get_subset ( ) const [inline]

gets subset flag

Definition at line 46 of file storexml.hpp.

References _subset.

Referenced by byte_manager::get_subset().

size_t byte_source::get_version ( ) const [inline]

gets version

Definition at line 53 of file storexml.hpp.

References _version.

Referenced by byte_manager::get_version(), and parseXMLDeclInfo().

size_t byte_source::get_xml_size ( ) const [inline]

xml size

Definition at line 60 of file storexml.hpp.

References _xml_size.

size_t byte_source::get_buffer_pos ( ) const [inline, protected]

gets buffer position

void byte_source::reset_buffer ( ) [protected]

resets internal buffer

Definition at line 97 of file storexml.cpp.

References _active_store, _buffer_pos, _char_counter, _encodingSchema, _end, _line_counter, _list_allocator, _pos_counter, _standalone, _store_allocator, _symbol, _used_store, _version, _xml_size, AUTO, base_list< T >::clear(), os_minus_one, and byte_allocator::reset().

Member Data Documentation

byte_allocator* byte_source::_depot_allocator [protected]

depot allocator

Definition at line 217 of file storexml.h.

Referenced by byte_source(), and ~byte_source().

byte_allocator* byte_source::_convert_allocator [protected]

conversion allocator

Definition at line 218 of file storexml.h.

Referenced by byte_source(), and ~byte_source().

byte_allocator* byte_source::_store_allocator [protected]

temporary store allocator

Definition at line 219 of file storexml.h.

Referenced by byte_source(), push(), reset_buffer(), and ~byte_source().

byte_allocator* byte_source::_list_allocator [protected]

store buffers allocator

Definition at line 220 of file storexml.h.

Referenced by byte_source(), go_shopping(), push(), reset_buffer(), and ~byte_source().

mem_pool_t& byte_source::_small_pool [protected]

small memory pool

Definition at line 221 of file storexml.h.

Referenced by byte_source(), parseXMLDeclInfo(), and ~byte_source().

mem_pool_t& byte_source::_big_pool [protected]

big memory pool

Definition at line 222 of file storexml.h.

Referenced by byte_source(), parseXMLDeclInfo(), and ~byte_source().

const size_t byte_source::_xml_size [protected]

xml size

Definition at line 223 of file storexml.h.

Referenced by auto_convert(), byte_source(), convert_chars(), get_xml_size(), go_shopping(), parseXMLDeclInfo(), pick(), pop(), pull(), push(), reset_buffer(), and ~byte_source().

string_t byte_source::_url [protected]

the location of xml stream

Definition at line 224 of file storexml.h.

Referenced by get_location(), and stream_input_common::open().

bool byte_source::_subset [private]

subset flag

Definition at line 226 of file storexml.h.

Referenced by get_subset(), and parseXMLDeclInfo().

ub1_t* byte_source::_buffer [private]

current buffer

Definition at line 227 of file storexml.h.

Referenced by byte_source(), convert_chars(), go_shopping(), parseXMLDeclInfo(), pop(), pull(), push(), and taste_buffer().

ub1_t* byte_source::_convert_buffer [private]

convert buffer

Definition at line 228 of file storexml.h.

Referenced by byte_source(), convert_chars(), parseEncoding(), parseStandalone(), parseVersion(), and taste_buffer().

size_t byte_source::_buffer_pos [private]

current buffer position

Definition at line 229 of file storexml.h.

Referenced by auto_convert(), convert_chars(), go_shopping(), parseXMLDeclInfo(), pick(), pop(), pull(), push(), reset_buffer(), and taste_buffer().

ub1_t byte_source::_symbol [private]

current symbol

Definition at line 230 of file storexml.h.

Referenced by convert_chars(), go_shopping(), pick(), pop(), push(), reset_buffer(), and taste_buffer().

store_list_t byte_source::_active_store [private]

store buffers in use

Definition at line 231 of file storexml.h.

Referenced by go_shopping(), push(), and reset_buffer().

store_list_t byte_source::_used_store [private]

store buffers for reusing

Definition at line 232 of file storexml.h.

Referenced by go_shopping(), push(), and reset_buffer().

size_t byte_source::_line_counter [private]

line number

Definition at line 233 of file storexml.h.

Referenced by pop(), reset_buffer(), and throw_exception().

size_t byte_source::_char_counter [private]

char position (for multibyte chars)

Definition at line 234 of file storexml.h.

Referenced by pop(), pull(), reset_buffer(), and throw_exception().

size_t byte_source::_pos_counter [private]

byte position

Definition at line 235 of file storexml.h.

Referenced by current_pos(), pop(), pull(), push(), and reset_buffer().

encodingSchema byte_source::_encodingSchema [private]

encoding schema

Definition at line 236 of file storexml.h.

Referenced by convert_chars(), get_encoding(), go_shopping(), parseTextDecl(), parseXMLDecl(), parseXMLDeclInfo(), reset_buffer(), set_encoding(), and taste_buffer().

size_t byte_source::_version [private]

version

Definition at line 237 of file storexml.h.

Referenced by get_version(), parseTextDecl(), parseVersion(), parseXMLDecl(), parseXMLDeclInfo(), and reset_buffer().

size_t byte_source::_standalone [private]

standalone flag

Definition at line 238 of file storexml.h.

Referenced by get_standalone(), parseXMLDecl(), parseXMLDeclInfo(), and reset_buffer().

bool byte_source::_end [private]

end of stream flag

Definition at line 239 of file storexml.h.

Referenced by go_shopping(), pull(), and reset_buffer().

The documentation for this class was generated from the following files: