Class AbstractCharStreamScanner

    • Field Detail

      • buffer

        protected char[] buffer
        The internal buffer with character data.
      • offset

        protected int offset
        The start position in the buffer from where reading operations consumer data from.
      • limit

        protected int limit
        The limit in the buffer. If the offset has reached this position no further reading (not even the current offset) from the buffer is allowed.
    • Constructor Detail

      • AbstractCharStreamScanner

        public AbstractCharStreamScanner​(int capacity)
        The constructor.
        Parameters:
        capacity - the capacity of the internal buffer in chars.
      • AbstractCharStreamScanner

        public AbstractCharStreamScanner​(char[] buffer)
        The constructor.
        Parameters:
        buffer - the internal char[] buffer.
    • Method Detail

      • reset

        protected void reset()
        Resets the internal state.
      • append

        protected StringBuilder append​(StringBuilder builder,
                                       int start,
                                       int end)
        Parameters:
        builder - a local StringBuilder variable to be allocated lazily.
        start - the start index in the underlying buffer to append.
        end - the limit index in the underlying buffer pointing to the next position after the last character to append.
        Returns:
        the given StringBuilder if not null or otherwise a reused StringBuilder instance that has been reset.
      • getAppended

        protected String getAppended​(StringBuilder builder,
                                     int start,
                                     int end)
        Parameters:
        builder - the local StringBuilder instance where data may already have been appended to. May be null.
        start - the start index in the underlying buffer to append.
        end - the limit index in the underlying buffer pointing to the next position after the last character to append.
        Returns:
        the String with the underlying buffer data from start to end-1 potentially appended to the given StringBuilder if not null.
      • hasNext

        public boolean hasNext()
        Description copied from interface: CharStreamScanner
        This method determines if there is at least one more character available.
        Specified by:
        hasNext in interface CharStreamScanner
        Returns:
        true if there is at least one character available, false if the end of text (EOT) has been reached.
      • isEos

        public boolean isEos()
        Specified by:
        isEos in interface CharStreamScanner
        Returns:
        true if the end of stream (EOS) has been reached, false otherwise. If true (EOS) the internal buffer contains the entire rest of the data to scan in memory. If then also all data is consumed from the buffer, EOT has been reached. For instances of that are not backed by an underlying stream of data (like CharSequenceScanner) this method will always return true.
      • isEob

        protected boolean isEob()
        Returns:
        true if end of buffer (EOB) or in other words no data is available after the current buffer, false otherwise (e.g. if not EOS).
      • isEot

        public boolean isEot()
        Specified by:
        isEot in interface CharStreamScanner
        Returns:
        true if end of text (EOT) is known to have been reached, false otherwise. The returned result will be almost the same as !CharStreamScanner.hasNext() but this method will not modify the state of this scanner (read additional data, modify buffers, etc.). However, if the underlying stream is already consumed without returning -1 to signal EOS this method may return false even though the next call of CharStreamScanner.hasNext() may also return false.
      • fill

        protected boolean fill()
        Fills the internal buffer with further data (if available from underlying source such as a stream/reader). If the end of the stream has not been reached, all buffers should be filled now.
        Returns:
        true if data was filled, false if EOS.
      • next

        public char next()
        Description copied from interface: CharStreamScanner
        This method reads the current character and increments the index stepping to the next character. You need to check if a character is available before calling this method.
        Specified by:
        next in interface CharStreamScanner
        Returns:
        the current character.
      • peek

        public char peek()
        Description copied from interface: CharStreamScanner
        This method reads the current character without incrementing the index. You need to check if a character is available before calling this method.
        Specified by:
        peek in interface CharStreamScanner
        Returns:
        the current character.
      • forcePeek

        public char forcePeek()
        Description copied from interface: CharStreamScanner
        This method reads the current character without incrementing the index. If there is no character available this method will return 0 (the NULL character and NOT '0').
        Specified by:
        forcePeek in interface CharStreamScanner
        Returns:
        the current character or 0 if none is available.
      • readUntil

        public String readUntil​(char stop,
                                boolean acceptEot)
        Description copied from interface: CharStreamScanner
        This method reads all next characters until the given stop character or the end is reached.
        After the call of this method, the current index will point to the next character after the (first) stop character or to the end if NO such character exists.
        Specified by:
        readUntil in interface CharStreamScanner
        Parameters:
        stop - is the character to read until.
        acceptEot - if true EOT will be treated as stop, too.
        Returns:
        the string with all read characters excluding the stop character or null if there was no stop character and acceptEot is false.
      • readUntil

        public String readUntil​(char stop,
                                boolean acceptEot,
                                CharScannerSyntax syntax)
        Description copied from interface: CharStreamScanner
        This method reads all next characters until the given stop character or the end of the string to parse is reached. In advance to CharStreamScanner.readUntil(char, boolean), this method will scan the input using the given syntax which e.g. allows to escape the stop character.
        After the call of this method, the current index will point to the next character after the (first) stop character or to the end of the string if NO such character exists.
        Specified by:
        readUntil in interface CharStreamScanner
        Parameters:
        stop - is the character to read until.
        acceptEot - if true EOT will be treated as stop, too.
        syntax - contains the characters specific for the syntax to read.
        Returns:
        the string with all read characters excluding the stop character or null if there was no stop character.
        See Also:
        CharStreamScanner.readUntil(CharFilter, boolean, CharScannerSyntax)
      • readUntil

        public String readUntil​(char stop,
                                boolean acceptEot,
                                char escape)
        Description copied from interface: CharStreamScanner
        This method reads all next characters until the given (un-escaped) stop character or the end is reached.
        In advance to CharStreamScanner.readUntil(char, boolean), this method allows that the stop character may be used in the input-string by adding the given escape character. After the call of this method, the current index will point to the next character after the (first) stop character or to the end if NO such character exists.
        This method is especially useful when quoted strings should be parsed. E.g.:
         CharStreamScanner scanner = getScanner();
         doSomething();
         char c = scanner.CharStreamScanner.forceNext();
         if ((c == '"') || (c == '\'')) {
           char escape = c; // may also be something like '\'
           String quote = scanner.readUntil(c, false, escape)
         } else {
           doOtherThings();
         }
         
        Specified by:
        readUntil in interface CharStreamScanner
        Parameters:
        stop - is the character to read until.
        acceptEot - if true EOT will be treated as stop, too.
        escape - is the character used to escape the stop character. To add an occurrence of the escape character it has to be duplicated (occur twice). The escape character may also be equal to the stop character. If other regular characters are escaped the escape character is simply ignored.
        Returns:
        the string with all read characters excluding the stop character or null if there was no stop character and acceptEot is false.
      • readUntil

        public String readUntil​(CharFilter filter,
                                boolean acceptEot)
        Description copied from interface: CharStreamScanner
        This method reads all next characters until the first character accepted by the given filter or the end is reached.
        After the call of this method, the current index will point to the first accepted stop character or to the end if NO such character exists.
        Specified by:
        readUntil in interface CharStreamScanner
        Parameters:
        filter - is used to decide where to stop.
        acceptEot - if true if EOT should be treated like the stop character and the rest of the text will be returned, false otherwise (to return null if EOT was reached and the scanner has been consumed).
        Returns:
        the string with all read characters not accepted by the given CharFilter or null if there was no accepted character and acceptEot is false.
      • readUntil

        public String readUntil​(CharFilter stopFilter,
                                boolean acceptEot,
                                String stop,
                                boolean ignoreCase,
                                boolean trim)
        Description copied from interface: CharStreamScanner
        This method reads all next characters until the first character accepted by the given filter, the given stop String or the end is reached.
        After the call of this method, the current index will point to the first accepted stop character, or to the first character of the given stop String or to the end if NO such character exists.
        Specified by:
        readUntil in interface CharStreamScanner
        Parameters:
        stopFilter - is used to decide where to stop.
        acceptEot - if true if EOT should be treated like the stop character and the rest of the text will be returned, false otherwise (to return null if EOT was reached and the scanner has been consumed).
        stop - the String where to stop consuming data. Should be at least two characters long (otherwise accept by CharFilter instead).
        ignoreCase - - if true the case of the characters is ignored when compared with characters from stop String.
        trim - - true if the result should be trimmed, false otherwise.
        Returns:
        the string with all read characters not accepted by the given CharFilter or until the given stop String was detected. If EOT was reached without a stop signal the entire rest of the data is returned or null if acceptEot is false. Thre result will be trimmed if trim is true.
      • verifyLookahead

        protected void verifyLookahead​(String substring)
        Parameters:
        substring - the substring to match without consuming what requires a lookahead.
      • expectRestWithLookahead

        protected abstract boolean expectRestWithLookahead​(char[] stopChars,
                                                           boolean ignoreCase,
                                                           Runnable appender,
                                                           boolean skip)
        Parameters:
        stopChars - the stop String as char[]. If ignoreCase is true in lower case.
        ignoreCase - - true to (also) compare chars in lower case, false otherwise.
        appender - an optional lambda to run before shifting buffers to append data.
        skip - - true to update buffers and offset such that on success this scanner points after the expected stop String, false otherwise (to not consume any character in any case).
        Returns:
        true if the stop String (stopChars) was found and consumed, false otherwise (and no data consumed).
        See Also:
        CharStreamScanner.readUntil(CharFilter, boolean, String, boolean), skipOver(String, boolean, CharFilter)
      • require

        public void require​(char expected)
        Description copied from interface: CharStreamScanner
        This method verifies that the current character is equal to the given expected character.
        If the current character was as expected, the parser points to the next character. Otherwise an exception is thrown indicating the problem.
        Specified by:
        require in interface CharStreamScanner
        Parameters:
        expected - is the expected character.
      • require

        public void require​(String expected,
                            boolean ignoreCase)
        Description copied from interface: CharStreamScanner
        This method verifies that the expected string gets consumed from this scanner with respect to ignoreCase. Otherwise an exception is thrown indicating the problem.
        This method behaves functionally equivalent to the following code:
         if (!scanner.expectStrict(expected, ignoreCase)) {
           throw new IllegalStateException(...);
         }
         
        Specified by:
        require in interface CharStreamScanner
        Parameters:
        expected - is the expected string.
        ignoreCase - - if true the case of the characters is ignored during comparison.
      • expect

        public boolean expect​(char expected)
        Description copied from interface: CharStreamScanner
        This method checks that the current character is equal to the given expected character.
        If the current character was as expected, the parser points to the next character. Otherwise its position will remain unchanged.
        Specified by:
        expect in interface CharStreamScanner
        Parameters:
        expected - is the expected character.
        Returns:
        true if the current character is the same as expected, false otherwise.
      • expect

        public boolean expect​(String expected,
                              boolean ignoreCase)
        Description copied from interface: CharStreamScanner
        This method skips all next characters as long as they equal to the according character of the expected string.
        If a character differs this method stops and the parser points to the first character that differs from expected. Except for the latter circumstance, this method behaves like the following code:
         read(expected.length).equals[IgnoreCase](expected)
         
        ATTENTION:
        Be aware that if already the first character differs, this method will NOT change the state of the scanner. So take care NOT to produce infinity loops.
        Specified by:
        expect in interface CharStreamScanner
        Parameters:
        expected - is the expected string.
        ignoreCase - - if true the case of the characters is ignored when compared.
        Returns:
        true if the expected string was successfully consumed from this scanner, false otherwise.
      • readJavaStringLiteral

        public String readJavaStringLiteral​(boolean tolerant)
        Description copied from interface: CharStreamScanner
        Reads and parses a Java String literal value according to JLS 3.10.6.
        As a complex example for the input "Hi \"\176\477\579•∑\"\n" this scanner would return the String output Hi "~'7/9•∑" followed by a newline character.
        Specified by:
        readJavaStringLiteral in interface CharStreamScanner
        Parameters:
        tolerant - - true if invalid escape sequences should be tolerated (as '?'), false to throw an exception in such case.
        Returns:
        the parsed Java String literal value or null if not pointing to a String literal.
      • readJavaCharLiteral

        public Character readJavaCharLiteral​(boolean tolerant)
        Description copied from interface: CharStreamScanner
        Reads and parses a Java Character literal value according to JLS 3.10.6.
        Examples are given in the following table:
        literal result comment
        'a' a regular char
        '\'' ' escaped char
        '\176' ~ escaped octal representation
        '•' escaped unicode representation
        Specified by:
        readJavaCharLiteral in interface CharStreamScanner
        Parameters:
        tolerant - - true if an invalid char literal should be tolerated (as '?'), false to throw an exception in such case.
        Returns:
        the parsed Java String literal value or null if not pointing to a String literal.
      • read

        public String read​(int count)
        Description copied from interface: CharStreamScanner
        This method reads the number of next characters given by count and returns them as string. If there are less characters available the returned string will be shorter than count and only contain the available characters.
        Specified by:
        read in interface CharStreamScanner
        Parameters:
        count - is the number of characters to read. You may use Integer.MAX_VALUE to read until the end of data if the data-size is suitable.
        Returns:
        a string with the given number of characters or all available characters if less than count. Will be the empty string if no character is available at all.
      • readLong

        public long readLong​(int maxDigits)
                      throws NumberFormatException
        Description copied from interface: CharStreamScanner
        This method reads the long starting at the current position by reading as many Latin digits as available but at maximum the given maxDigits and returns its parsed value.
        ATTENTION:
        This method does NOT treat signs (+ or -) to do so, scan them yourself before and negate the result as needed.
        Specified by:
        readLong in interface CharStreamScanner
        Parameters:
        maxDigits - is the maximum number of digits that will be read. The value has to be positive (greater than zero). Use 19 or higher to be able to read any long number.
        Returns:
        the parsed number.
        Throws:
        NumberFormatException - if the current current position does NOT point to a number.
      • skip

        public int skip​(int count)
        Description copied from interface: CharStreamScanner
        This method skips the number of next characters given by count.
        Specified by:
        skip in interface CharStreamScanner
        Parameters:
        count - is the number of characters to skip. You may use Integer.MAX_VALUE to read until the end of data if the data-size is suitable.
        Returns:
        a to total number of characters that have been skipped. Typically equal to count. Will be less in case the end of the stream was reached.
      • skipUntil

        public boolean skipUntil​(char stop)
        Description copied from interface: CharStreamScanner
        This method skips all next characters until the given stop character or the end is reached. If the stop character was reached, this scanner will point to the next character after stop when this method returns.
        Specified by:
        skipUntil in interface CharStreamScanner
        Parameters:
        stop - is the character to read until.
        Returns:
        true if the first occurrence of the given stop character has been passed, false if there is no such character.
      • skipUntil

        public boolean skipUntil​(char stop,
                                 char escape)
        Description copied from interface: CharStreamScanner
        This method reads all next characters until the given stop character or the end of the string to parse is reached. In advance to CharStreamScanner.skipUntil(char), this method will read over the stop character if it is escaped with the given escape character.
        Specified by:
        skipUntil in interface CharStreamScanner
        Parameters:
        stop - is the character to read until.
        escape - is the character used to escape the stop character (e.g. '\').
        Returns:
        true if the first occurrence of the given stop character has been passed, false if there is no such character.
      • skipWhile

        public int skipWhile​(char c)
        Description copied from interface: CharStreamScanner
        This method reads all next characters that are identical to the character given by c.
        E.g. use readWhile(' ') to skip all blanks from the current index. After the call of this method, the current index will point to the next character that is different to the given character c or to the end if NO such character exists.
        Specified by:
        skipWhile in interface CharStreamScanner
        Parameters:
        c - is the character to read over.
        Returns:
        the number of characters that have been skipped.
      • skipWhile

        public int skipWhile​(CharFilter filter,
                             int max)
        Description copied from interface: CharStreamScanner
        This method reads all next characters that are accepted by the given filter.
        After the call of this method, the current index will point to the next character that was NOT accepted by the given filter. If the next max characters or the characters left until the end of this scanner are accepted, only that amount of characters are skipped.
        Specified by:
        skipWhile in interface CharStreamScanner
        Parameters:
        filter - is used to decide which characters should be accepted.
        max - is the maximum number of characters that may be skipped.
        Returns:
        the number of skipped characters.
        See Also:
        CharStreamScanner.skipWhile(char)
      • skipOver

        public boolean skipOver​(String substring,
                                boolean ignoreCase,
                                CharFilter stopFilter)
        Description copied from interface: CharStreamScanner
        This method consumes all next characters until the given substring has been detected, a character was accepted by the given CharFilter or EOT was reached.
        After the call of this method this scanner will point to the next character after the first occurrence of substring, to the stop character or to EOT.
        Specified by:
        skipOver in interface CharStreamScanner
        Parameters:
        substring - is the substring to search and skip over starting at the current index.
        ignoreCase - - if true the case of the characters is ignored when compared with characters from substring.
        stopFilter - is the filter used to detect stop characters. If such character was detected, the skip is stopped and the parser points to the character after the stop character. The substring should NOT contain a stop character.
        Returns:
        true if the given substring occurred and has been passed and false if a stop character has been detected or the end of the string has been reached without any occurrence of the given substring or stop character.
      • readWhile

        public String readWhile​(CharFilter filter,
                                int max)
        Description copied from interface: CharStreamScanner
        This method reads all next characters that are accepted by the given filter.
        After the call of this method, the current index will point to the next character that was NOT accepted by the given filter. If the next max characters or the characters left until the end of this scanner are accepted, only that amount of characters are skipped.
        Specified by:
        readWhile in interface CharStreamScanner
        Parameters:
        filter - is used to decide which characters should be accepted.
        max - is the maximum number of characters that should be read.
        Returns:
        a string with all characters accepted by the given filter limited to the length of max and the end of this scanner. Will be the empty string if no character was accepted.
        See Also:
        CharStreamScanner.skipWhile(char)