RDoc::TokenStream

A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the start_collecting_tokens method, followed by calls to add_token and pop_token.

Public Instance Methods

add_token(*tokens) click to toggle source

Alias for: add_tokens

add_tokens(*tokens) click to toggle source

Adds tokens to the collected tokens

    # File lib/rdoc/tokenstream.rb, line 15
15:   def add_tokens(*tokens)
16:     tokens.flatten.each { |token| @token_stream << token }
17:   end

Also aliased as: add_token

collect_tokens() click to toggle source

Starts collecting tokens

    # File lib/rdoc/tokenstream.rb, line 24
24:   def collect_tokens
25:     @token_stream = []
26:   end

Also aliased as: start_collecting_tokens

pop_token() click to toggle source

Remove the last token from the collected tokens

    # File lib/rdoc/tokenstream.rb, line 33
33:   def pop_token
34:     @token_stream.pop
35:   end

start_collecting_tokens() click to toggle source

Alias for: collect_tokens

token_stream() click to toggle source

Current token stream

    # File lib/rdoc/tokenstream.rb, line 40
40:   def token_stream
41:     @token_stream
42:   end

tokens_to_s() click to toggle source

Returns a string representation of the token stream

    # File lib/rdoc/tokenstream.rb, line 47
47:   def tokens_to_s
48:     token_stream.map { |token| token.text }.join ''
49:   end

Home Classes Methods

In Files

Methods

Files

Class Index

RDoc::TokenStream

Public Instance Methods