RDoc::Parser::Ruby

Extracts code elements from a source file returning a TopLevel object containing the constituent file elements.

This file is based on rtags

RubyParser understands how to document:

classes
modules
methods
constants
aliases
private, public, protected
private_class_function, public_class_function
module_function
attr, attr_reader, attr_writer, attr_accessor
extra accessors given on the command line
metaprogrammed methods
require
include

Method Arguments

The parser extracts the arguments from the method definition. You can override this with a custom argument definition using the :call-seq: directive:

  ##
  # This method can be called with a range or an offset and length
  #
  # :call-seq:
  #   my_method(Range)
  #   my_method(offset, length)

  def my_method(*args)
  end

The parser extracts yield expressions from method bodies to gather the yielded argument names. If your method manually calls a block instead of yielding or you want to override the discovered argument names use the :yields: directive:

  ##
  # My method is awesome

  def my_method(&block) # :yields: happy, times
    block.call 1, 2
  end

Metaprogrammed Methods

To pick up a metaprogrammed method, the parser looks for a comment starting with ’##’ before an identifier:

  ##
  # This is a meta-programmed method!

  add_my_method :meta_method, :arg1, :arg2

The parser looks at the token after the identifier to determine the name, in this example, :meta_method. If a name cannot be found, a warning is printed and ‘unknown is used.

You can force the name of a method using the :method: directive:

  ##
  # :method: woo_hoo!

By default, meta-methods are instance methods. To indicate that a method is a singleton method instead use the :singleton-method: directive:

  ##
  # :singleton-method:

You can also use the :singleton-method: directive with a name:

  ##
  # :singleton-method: woo_hoo!

Additionally you can mark a method as an attribute by using :attr:, :attr_reader:, :attr_writer: or :attr_accessor:. Just like for :method:, the name is optional.

  ##
  # :attr_reader: my_attr_name

Hidden methods and attributes

You can provide documentation for methods that don’t appear using the :method:, :singleton-method: and :attr: directives:

  ##
  # :attr_writer: ghost_writer
  # There is an attribute here, but you can't see it!

  ##
  # :method: ghost_method
  # There is a method here, but you can't see it!

  ##
  # this is a comment for a regular method

  def regular_method() end

Note that by default, the :method: directive will be ignored if there is a standard rdocable item following it.

Constants

NORMAL: RDoc::NormalClass type
SINGLE: RDoc::SingleClass type

Public Class Methods

new(top_level, file_name, content, options, stats) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 165
165:   def initialize(top_level, file_name, content, options, stats)
166:     super
167: 
168:     @size = 0
169:     @token_listeners = nil
170:     @scanner = RDoc::RubyLex.new content, @options
171:     @scanner.exception_on_syntax_error = false
172:     @prev_seek = nil
173: 
174:     reset
175:   end

Public Instance Methods

collect_first_comment() click to toggle source

Look for the first comment in a file that isn’t a shebang line.

     # File lib/rdoc/parser/ruby.rb, line 180
180:   def collect_first_comment
181:     skip_tkspace
182:     comment = ''
183:     first_line = true
184: 
185:     tk = get_tk
186: 
187:     while TkCOMMENT === tk
188:       if first_line and tk.text =~ /\A#!/ then
189:         skip_tkspace
190:         tk = get_tk
191:       elsif first_line and tk.text =~ /\A#\s*-\*-/ then
192:         first_line = false
193:         skip_tkspace
194:         tk = get_tk
195:       else
196:         first_line = false
197:         comment << tk.text << "\n"
198:         tk = get_tk
199: 
200:         if TkNL === tk then
201:           skip_tkspace false
202:           tk = get_tk
203:         end
204:       end
205:     end
206: 
207:     unget_tk tk
208: 
209:     comment
210:   end

error(msg) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 212
212:   def error(msg)
213:     msg = make_message msg
214:     $stderr.puts msg
215:     exit false
216:   end

extract_call_seq(comment, meth) click to toggle source

Look for a ‘call-seq’ in the comment, and override the normal parameter stuff

     # File lib/rdoc/parser/ruby.rb, line 222
222:   def extract_call_seq(comment, meth)
223:     if comment.sub!(/:?call-seq:(.*?)^\s*\#?\s*$/, '') then
224:       seq = $1
225:       seq.gsub!(/^\s*\#\s*/, '')
226:       meth.call_seq = seq
227:     end
228: 
229:     meth
230:   end

get_bool() click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 232
232:   def get_bool
233:     skip_tkspace
234:     tk = get_tk
235:     case tk
236:     when TkTRUE
237:       true
238:     when TkFALSE, TkNIL
239:       false
240:     else
241:       unget_tk tk
242:       true
243:     end
244:   end

get_class_or_module(container) click to toggle source

Look for the name of a class of module (optionally with a leading	or
with	separated named) and return the ultimate name and container

     # File lib/rdoc/parser/ruby.rb, line 250
250:   def get_class_or_module(container)
251:     skip_tkspace
252:     name_t = get_tk
253: 
254:     # class ::A -> A is in the top level
255:     case name_t
256:     when TkCOLON2, TkCOLON3 then # bug
257:       name_t = get_tk
258:       container = @top_level
259:     end
260: 
261:     skip_tkspace false
262: 
263:     while TkCOLON2 === peek_tk do
264:       prev_container = container
265:       container = container.find_module_named name_t.name
266:       unless container then
267:         container = prev_container.add_module RDoc::NormalModule, name_t.name
268:       end
269:       get_tk
270:       name_t = get_tk
271:     end
272:     skip_tkspace false
273:     return [container, name_t]
274:   end

get_class_specification() click to toggle source

Return a superclass, which can be either a constant of an expression

     # File lib/rdoc/parser/ruby.rb, line 279
279:   def get_class_specification
280:     tk = get_tk
281:     return "self" if TkSELF === tk
282: 
283:     res = ""
284:     while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
285:       res += tk.name
286:       tk = get_tk
287:     end
288: 
289:     unget_tk(tk)
290:     skip_tkspace false
291: 
292:     get_tkread # empty out read buffer
293: 
294:     tk = get_tk
295: 
296:     case tk
297:     when TkNL, TkCOMMENT, TkSEMICOLON then
298:       unget_tk(tk)
299:       return res
300:     end
301: 
302:     res += parse_call_parameters(tk)
303:     res
304:   end

get_constant() click to toggle source

Parse a constant, which might be qualified by one or more class or module names

     # File lib/rdoc/parser/ruby.rb, line 310
310:   def get_constant
311:     res = ""
312:     skip_tkspace false
313:     tk = get_tk
314: 
315:     while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
316:       res += tk.name
317:       tk = get_tk
318:     end
319: 
320: #      if res.empty?
321: #        warn("Unexpected token #{tk} in constant")
322: #      end
323:     unget_tk(tk)
324:     res
325:   end

get_constant_with_optional_parens() click to toggle source

Get a constant that may be surrounded by parens

     # File lib/rdoc/parser/ruby.rb, line 330
330:   def get_constant_with_optional_parens
331:     skip_tkspace false
332:     nest = 0
333:     while TkLPAREN === (tk = peek_tk) or TkfLPAREN === tk do
334:       get_tk
335:       skip_tkspace
336:       nest += 1
337:     end
338: 
339:     name = get_constant
340: 
341:     while nest > 0
342:       skip_tkspace
343:       tk = get_tk
344:       nest -= 1 if TkRPAREN === tk
345:     end
346: 
347:     name
348:   end

get_symbol_or_name() click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 350
350:   def get_symbol_or_name
351:     tk = get_tk
352:     case tk
353:     when TkSYMBOL then
354:       text = tk.text.sub(/^:/, '')
355: 
356:       if TkASSIGN === peek_tk then
357:         get_tk
358:         text << '='
359:       end
360: 
361:       text
362:     when TkId, TkOp then
363:       tk.name
364:     when TkSTRING, TkDSTRING then
365:       tk.text
366:     else
367:       raise RDoc::Error, "Name or symbol expected (got #{tk})"
368:     end
369:   end

look_for_directives_in(context, comment) click to toggle source

Look for directives in a normal comment block:

  # :stopdoc:
  # Don't display comment from this point forward

This routine modifies it’s parameter

     # File lib/rdoc/parser/ruby.rb, line 379
379:   def look_for_directives_in(context, comment)
380:     preprocess = RDoc::Markup::PreProcess.new @file_name, @options.rdoc_include
381: 
382:     preprocess.handle comment, context do |directive, param|
383:       case directive
384:       when 'enddoc' then
385:         throw :enddoc
386:       when 'main' then
387:         @options.main_page = param
388:         ''
389:       when 'method', 'singleton-method',
390:            'attr', 'attr_accessor', 'attr_reader', 'attr_writer' then
391:         false # handled elsewhere
392:       when 'section' then
393:         context.set_current_section param, comment
394:         comment.replace ''
395:         break
396:       when 'startdoc' then
397:         context.start_doc
398:         context.force_documentation = true
399:         ''
400:       when 'stopdoc' then
401:         context.stop_doc
402:         ''
403:       when 'title' then
404:         @options.title = param
405:         ''
406:       end
407:     end
408: 
409:     remove_private_comments comment
410:   end

make_message(message) click to toggle source

Adds useful info about the parser to message

     # File lib/rdoc/parser/ruby.rb, line 415
415:   def make_message message
416:     prefix = "#{@file_name}:"
417: 
418:     prefix << "#{@scanner.line_no}:#{@scanner.char_no}:" if @scanner
419: 
420:     "#{prefix} #{message}"
421:   end

parse_alias(context, single, tk, comment) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 478
478:   def parse_alias(context, single, tk, comment)
479:     skip_tkspace
480:     if TkLPAREN === peek_tk then
481:       get_tk
482:       skip_tkspace
483:     end
484:     new_name = get_symbol_or_name
485: 
486:     @scanner.instance_eval { @lex_state = EXPR_FNAME }
487: 
488:     skip_tkspace
489:     if TkCOMMA === peek_tk then
490:       get_tk
491:       skip_tkspace
492:     end
493: 
494:     begin
495:       old_name = get_symbol_or_name
496:     rescue RDoc::Error
497:       return
498:     end
499: 
500:     al = RDoc::Alias.new get_tkread, old_name, new_name, comment
501:     al.singleton = SINGLE == single
502:     read_documentation_modifiers al, RDoc::ATTR_MODIFIERS
503: 
504:     context.add_alias al if al.document_self
505:     @stats.add_alias al
506: 
507:     al
508:   end

parse_attr(context, single, tk, comment) click to toggle source

Creates an RDoc::Attr for the name following tk, setting the comment to comment.

     # File lib/rdoc/parser/ruby.rb, line 427
427:   def parse_attr(context, single, tk, comment)
428:     args = parse_symbol_arg 1
429:     if args.size > 0
430:       name = args[0]
431:       rw = "R"
432:       skip_tkspace false
433:       tk = get_tk
434:       if TkCOMMA === tk then
435:         rw = "RW" if get_bool
436:       else
437:         unget_tk tk
438:       end
439:       att = RDoc::Attr.new get_tkread, name, rw, comment
440:       read_documentation_modifiers att, RDoc::ATTR_MODIFIERS
441:       if att.document_self
442:         context.add_attribute(att)
443:       end
444:     else
445:       warn("'attr' ignored - looks like a variable")
446:     end
447:   end

parse_attr_accessor(context, single, tk, comment) click to toggle source

Creates an RDoc::Attr for each attribute listed after tk, setting the comment for each to comment.

     # File lib/rdoc/parser/ruby.rb, line 453
453:   def parse_attr_accessor(context, single, tk, comment)
454:     args = parse_symbol_arg
455:     read = get_tkread
456:     rw = "?"
457: 
458:     # TODO If nodoc is given, don't document any of them
459: 
460:     tmp = RDoc::CodeObject.new
461:     read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS
462:     return unless tmp.document_self
463: 
464:     case tk.name
465:     when "attr_reader"   then rw = "R"
466:     when "attr_writer"   then rw = "W"
467:     when "attr_accessor" then rw = "RW"
468:     else
469:       rw = '?'
470:     end
471: 
472:     for name in args
473:       att = RDoc::Attr.new get_tkread, name, rw, comment
474:       context.add_attribute att
475:     end
476:   end

parse_call_parameters(tk) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 510
510:   def parse_call_parameters(tk)
511:     end_token = case tk
512:                 when TkLPAREN, TkfLPAREN
513:                   TkRPAREN
514:                 when TkRPAREN
515:                   return ""
516:                 else
517:                   TkNL
518:                 end
519:     nest = 0
520: 
521:     loop do
522:       case tk
523:       when TkSEMICOLON
524:         break
525:       when TkLPAREN, TkfLPAREN
526:         nest += 1
527:       when end_token
528:         if end_token == TkRPAREN
529:           nest -= 1
530:           break if @scanner.lex_state == EXPR_END and nest <= 0
531:         else
532:           break unless @scanner.continue
533:         end
534:       when TkCOMMENT
535:         unget_tk(tk)
536:         break
537:       when nil then
538:         break
539:       end
540:       tk = get_tk
541:     end
542:     res = get_tkread.tr("\n", " ").strip
543:     res = "" if res == ";"
544:     res
545:   end

parse_class(container, single, tk, comment) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 547
547:   def parse_class(container, single, tk, comment)
548:     container, name_t = get_class_or_module container
549: 
550:     case name_t
551:     when TkCONSTANT
552:       name = name_t.name
553:       superclass = "Object"
554: 
555:       if TkLT === peek_tk then
556:         get_tk
557:         skip_tkspace
558:         superclass = get_class_specification
559:         superclass = "<unknown>" if superclass.empty?
560:       end
561: 
562:       cls_type = single == SINGLE ? RDoc::SingleClass : RDoc::NormalClass
563:       cls = container.add_class cls_type, name, superclass
564: 
565:       read_documentation_modifiers cls, RDoc::CLASS_MODIFIERS
566:       cls.record_location @top_level
567:       cls.comment = comment
568: 
569:       @stats.add_class cls
570: 
571:       parse_statements cls
572:     when TkLSHFT
573:       case name = get_class_specification
574:       when "self", container.name
575:         parse_statements container, SINGLE
576:       else
577:         other = RDoc::TopLevel.find_class_named name
578: 
579:         unless other then
580:           other = container.add_module RDoc::NormalModule, name
581:           other.record_location @top_level
582:           other.comment = comment
583:         end
584: 
585:         @stats.add_class other
586: 
587:         read_documentation_modifiers other, RDoc::CLASS_MODIFIERS
588:         parse_statements(other, SINGLE)
589:       end
590: 
591:     else
592:       warn("Expected class name or '<<'. Got #{name_t.class}: #{name_t.text.inspect}")
593:     end
594:   end

parse_comment(container, tk, comment) click to toggle source

Generates an RDoc::Method or RDoc::Attr from comment by looking for :method: or :attr: directives in comment.

     # File lib/rdoc/parser/ruby.rb, line 675
675:   def parse_comment(container, tk, comment)
676:     line_no = tk.line_no
677:     column  = tk.char_no
678: 
679:     singleton = !!comment.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')
680: 
681:     # REFACTOR
682:     if comment.sub!(/^# +:?method: *(\S*).*?\n/, '') then
683:       name = $1 unless $1.empty?
684: 
685:       meth = RDoc::GhostMethod.new get_tkread, name
686:       meth.singleton = singleton
687: 
688:       meth.start_collecting_tokens
689:       indent = TkSPACE.new nil, 1, 1
690:       indent.set_text " " * column
691: 
692:       position_comment = TkCOMMENT.new nil, line_no, 1
693:       position_comment.set_text "# File #{@top_level.absolute_name}, line #{line_no}"
694:       meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
695: 
696:       meth.params = ''
697: 
698:       extract_call_seq comment, meth
699: 
700:       return unless meth.name
701: 
702:       container.add_method meth if meth.document_self
703: 
704:       meth.comment = comment
705: 
706:       @stats.add_method meth
707:     elsif comment.sub!(/# +:?(attr(_reader|_writer|_accessor)?:) *(\S*).*?\n/, '') then
708:       rw = case $1
709:            when 'attr_reader' then 'R'
710:            when 'attr_writer' then 'W'
711:            else 'RW'
712:            end
713: 
714:       name = $3 unless $3.empty?
715: 
716:       att = RDoc::Attr.new get_tkread, name, rw, comment
717:       container.add_attribute att
718: 
719:       @stats.add_method att
720:     end
721:   end

parse_constant(container, tk, comment) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 596
596:   def parse_constant(container, tk, comment)
597:     name = tk.name
598:     skip_tkspace false
599:     eq_tk = get_tk
600: 
601:     unless TkASSIGN === eq_tk then
602:       unget_tk eq_tk
603:       return
604:     end
605: 
606:     nest = 0
607:     get_tkread
608: 
609:     tk = get_tk
610: 
611:     if TkGT === tk then
612:       unget_tk tk
613:       unget_tk eq_tk
614:       return
615:     end
616: 
617:     rhs_name = ''
618: 
619:     loop do
620:       case tk
621:       when TkSEMICOLON then
622:         break
623:       when TkLPAREN, TkfLPAREN, TkLBRACE, TkLBRACK, TkDO, TkIF, TkUNLESS,
624:            TkCASE then
625:         nest += 1
626:       when TkRPAREN, TkRBRACE, TkRBRACK, TkEND then
627:         nest -= 1
628:       when TkCOMMENT then
629:         if nest <= 0 && @scanner.lex_state == EXPR_END
630:           unget_tk tk
631:           break
632:         end
633:       when TkCONSTANT then
634:         rhs_name << tk.name
635: 
636:         if nest <= 0 and TkNL === peek_tk then
637:           mod = if rhs_name =~ /^::/ then
638:                   RDoc::TopLevel.find_class_or_module rhs_name
639:                 else
640:                   container.find_module_named rhs_name
641:                 end
642: 
643:           container.add_module_alias mod, name if mod
644:           get_tk # TkNL
645:           break
646:         end
647:       when TkNL then
648:         if nest <= 0 &&
649:            (@scanner.lex_state == EXPR_END || !@scanner.continue) then
650:           unget_tk tk
651:           break
652:         end
653:       when TkCOLON2, TkCOLON3 then
654:         rhs_name << '::'
655:       when nil then
656:         break
657:       end
658:       tk = get_tk
659:     end
660: 
661:     res = get_tkread.tr("\n", " ").strip
662:     res = "" if res == ";"
663: 
664:     con = RDoc::Constant.new name, res, comment
665:     read_documentation_modifiers con, RDoc::CONSTANT_MODIFIERS
666: 
667:     @stats.add_constant con
668:     container.add_constant con if con.document_self
669:   end

parse_include(context, comment) click to toggle source

     # File lib/rdoc/parser/ruby.rb, line 723
723:   def parse_include(context, comment)
724:     loop do
725:       skip_tkspace_comment
726: 
727:       name = get_constant_with_optional_parens
728:       context.add_include RDoc::Include.new(name, comment) unless name.empty?
729: 
730:       return unless TkCOMMA === peek_tk
731:       get_tk
732:     end
733:   end

parse_meta_attr(context, single, tk, comment) click to toggle source

Parses a meta-programmed attribute and creates an RDoc::Attr.

To create foo and bar attributes on class C with comment “My attributes”:

  class C

    ##
    # :attr:
    #
    # My attributes

    my_attr :foo, :bar

  end

To create a foo attribute on class C with comment “My attribute”:

  class C

    ##
    # :attr: foo
    #
    # My attribute

    my_attr :foo, :bar

  end

     # File lib/rdoc/parser/ruby.rb, line 764
764:   def parse_meta_attr(context, single, tk, comment)
765:     args = parse_symbol_arg
766:     read = get_tkread
767:     rw = "?"
768: 
769:     # If nodoc is given, don't document any of them
770: 
771:     tmp = RDoc::CodeObject.new
772:     read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS
773:     return unless tmp.document_self
774: 
775:     if comment.sub!(/^# +:?(attr(_reader|_writer|_accessor)?): *(\S*).*?\n/, '') then
776:       rw = case $1
777:            when 'attr_reader' then 'R'
778:            when 'attr_writer' then 'W'
779:            else 'RW'
780:            end
781:       name = $3 unless $3.empty?
782:     end
783: 
784:     if name then
785:       att = RDoc::Attr.new get_tkread, name, rw, comment
786:       context.add_attribute att
787:     else
788:       args.each do |attr_name|
789:         att = RDoc::Attr.new get_tkread, attr_name, rw, comment
790:         context.add_attribute att
791:       end
792:     end
793:   end

parse_meta_method(container, single, tk, comment) click to toggle source

Parses a meta-programmed method

     # File lib/rdoc/parser/ruby.rb, line 798
798:   def parse_meta_method(container, single, tk, comment)
799:     line_no = tk.line_no
800:     column  = tk.char_no
801: 
802:     start_collecting_tokens
803:     add_token tk
804:     add_token_listener self
805: 
806:     skip_tkspace false
807: 
808:     singleton = !!comment.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')
809: 
810:     if comment.sub!(/^# +:?method: *(\S*).*?\n/, '') then
811:       name = $1 unless $1.empty?
812:     end
813: 
814:     if name.nil? then
815:       name_t = get_tk
816:       case name_t
817:       when TkSYMBOL then
818:         name = name_t.text[1..1]
819:       when TkSTRING then
820:         name = name_t.value[1..2]
821:       when TkASSIGN then # ignore
822:         remove_token_listener self
823:         return
824:       else
825:         warn "unknown name token #{name_t.inspect} for meta-method '#{tk.name}'"
826:         name = 'unknown'
827:       end
828:     end
829: 
830:     meth = RDoc::MetaMethod.new get_tkread, name
831:     meth.singleton = singleton
832: 
833:     remove_token_listener self
834: 
835:     meth.start_collecting_tokens
836:     indent = TkSPACE.new nil, 1, 1
837:     indent.set_text " " * column
838: 
839:     position_comment = TkCOMMENT.new nil, line_no, 1
840:     position_comment.value = "# File #{@top_level.absolute_name}, line #{line_no}"
841:     meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
842:     meth.add_tokens @token_stream
843: 
844:     token_listener meth do
845:       meth.params = ''
846: 
847:       extract_call_seq comment, meth
848: 
849:       container.add_method meth if meth.document_self
850: 
851:       last_tk = tk
852: 
853:       while tk = get_tk do
854:         case tk
855:         when TkSEMICOLON then
856:           break
857:         when TkNL then
858:           break unless last_tk and TkCOMMA === last_tk
859:         when TkSPACE then
860:           # expression continues
861:         else
862:           last_tk = tk
863:         end
864:       end
865:     end
866: 
867:     meth.comment = comment
868: 
869:     @stats.add_method meth
870:   end

parse_method(container, single, tk, comment) click to toggle source

Parses a normal method defined by def

      # File lib/rdoc/parser/ruby.rb, line 875
 875:   def parse_method(container, single, tk, comment)
 876:     added_container = nil
 877:     meth = nil
 878:     name = nil
 879:     line_no = tk.line_no
 880:     column  = tk.char_no
 881: 
 882:     start_collecting_tokens
 883:     add_token tk
 884: 
 885:     token_listener self do
 886:       @scanner.instance_eval do @lex_state = EXPR_FNAME end
 887: 
 888:       skip_tkspace false
 889:       name_t = get_tk
 890:       back_tk = skip_tkspace
 891:       meth = nil
 892:       added_container = false
 893: 
 894:       dot = get_tk
 895:       if TkDOT === dot or TkCOLON2 === dot then
 896:         @scanner.instance_eval do @lex_state = EXPR_FNAME end
 897:         skip_tkspace
 898:         name_t2 = get_tk
 899: 
 900:         case name_t
 901:         when TkSELF, TkMOD then
 902:           name = name_t2.name
 903:         when TkCONSTANT then
 904:           name = name_t2.name
 905:           prev_container = container
 906:           container = container.find_module_named(name_t.name)
 907:           unless container then
 908:             added_container = true
 909:             obj = name_t.name.split("::").inject(Object) do |state, item|
 910:               state.const_get(item)
 911:             end rescue nil
 912: 
 913:             type = obj.class == Class ? RDoc::NormalClass : RDoc::NormalModule
 914: 
 915:             unless [Class, Module].include?(obj.class) then
 916:               warn("Couldn't find #{name_t.name}. Assuming it's a module")
 917:             end
 918: 
 919:             if type == RDoc::NormalClass then
 920:               sclass = obj.superclass ? obj.superclass.name : nil
 921:               container = prev_container.add_class type, name_t.name, sclass
 922:             else
 923:               container = prev_container.add_module type, name_t.name
 924:             end
 925: 
 926:             container.record_location @top_level
 927:           end
 928:         when TkIDENTIFIER, TkIVAR then
 929:           dummy = RDoc::Context.new
 930:           dummy.parent = container
 931:           skip_method dummy
 932:           return
 933:         else
 934:           warn "unexpected method name token #{name_t.inspect}"
 935:           # break
 936:           skip_method container
 937:           return
 938:         end
 939: 
 940:         meth = RDoc::AnyMethod.new(get_tkread, name)
 941:         meth.singleton = true
 942:       else
 943:         unget_tk dot
 944:         back_tk.reverse_each do |token|
 945:           unget_tk token
 946:         end
 947: 
 948:         name = case name_t
 949:                when TkSTAR, TkAMPER then
 950:                  name_t.text
 951:                else
 952:                  unless name_t.respond_to? :name then
 953:                    warn "expected method name token, . or ::, got #{name_t.inspect}"
 954:                    skip_method container
 955:                    return
 956:                  end
 957:                  name_t.name
 958:                end
 959: 
 960:         meth = RDoc::AnyMethod.new get_tkread, name
 961:         meth.singleton = (single == SINGLE)
 962:       end
 963:     end
 964: 
 965:     meth.start_collecting_tokens
 966:     indent = TkSPACE.new nil, 1, 1
 967:     indent.set_text " " * column
 968: 
 969:     token = TkCOMMENT.new nil, line_no, 1
 970:     token.set_text "# File #{@top_level.absolute_name}, line #{line_no}"
 971:     meth.add_tokens [token, NEWLINE_TOKEN, indent]
 972:     meth.add_tokens @token_stream
 973: 
 974:     token_listener meth do
 975:       @scanner.instance_eval do @continue = false end
 976:       parse_method_parameters meth
 977: 
 978:       if meth.document_self then
 979:         container.add_method meth
 980:       elsif added_container then
 981:         container.document_self = false
 982:       end
 983: 
 984:       # Having now read the method parameters and documentation modifiers, we
 985:       # now know whether we have to rename #initialize to ::new
 986: 
 987:       if name == "initialize" && !meth.singleton then
 988:         if meth.dont_rename_initialize then
 989:           meth.visibility = :protected
 990:         else
 991:           meth.singleton = true
 992:           meth.name = "new"
 993:           meth.visibility = :public
 994:         end
 995:       end
 996: 
 997:       parse_statements container, single, meth
 998:     end
 999: 
1000:     extract_call_seq comment, meth
1001: 
1002:     meth.comment = comment
1003: 
1004:     @stats.add_method meth
1005:   end

parse_method_or_yield_parameters(method = nil, modifiers = RDoc::METHOD_MODIFIERS) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1007
1007:   def parse_method_or_yield_parameters(method = nil,
1008:                                        modifiers = RDoc::METHOD_MODIFIERS)
1009:     skip_tkspace false
1010:     tk = get_tk
1011: 
1012:     # Little hack going on here. In the statement
1013:     #  f = 2*(1+yield)
1014:     # We see the RPAREN as the next token, so we need
1015:     # to exit early. This still won't catch all cases
1016:     # (such as "a = yield + 1"
1017:     end_token = case tk
1018:                 when TkLPAREN, TkfLPAREN
1019:                   TkRPAREN
1020:                 when TkRPAREN
1021:                   return ""
1022:                 else
1023:                   TkNL
1024:                 end
1025:     nest = 0
1026: 
1027:     loop do
1028:       case tk
1029:       when TkSEMICOLON then
1030:         break
1031:       when TkLBRACE then
1032:         nest += 1
1033:       when TkRBRACE then
1034:         # we might have a.each {|i| yield i }
1035:         unget_tk(tk) if nest.zero?
1036:         nest -= 1
1037:         break if nest <= 0
1038:       when TkLPAREN, TkfLPAREN then
1039:         nest += 1
1040:       when end_token then
1041:         if end_token == TkRPAREN
1042:           nest -= 1
1043:           break if @scanner.lex_state == EXPR_END and nest <= 0
1044:         else
1045:           break unless @scanner.continue
1046:         end
1047:       when method && method.block_params.nil? && TkCOMMENT then
1048:         unget_tk tk
1049:         read_documentation_modifiers method, modifiers
1050:         @read.pop
1051:       when TkCOMMENT then
1052:         @read.pop
1053:       when nil then
1054:         break
1055:       end
1056:       tk = get_tk
1057:     end
1058: 
1059:     res = get_tkread.gsub(/\s+/, ' ').strip
1060:     res = '' if res == ';'
1061:     res
1062:   end

parse_method_parameters(method) click to toggle source

Capture the method’s parameters. Along the way, look for a comment containing:

   # yields: ....

and add this as the block_params for the method

      # File lib/rdoc/parser/ruby.rb, line 1072
1072:   def parse_method_parameters(method)
1073:     res = parse_method_or_yield_parameters method
1074: 
1075:     res = "(#{res})" unless res =~ /\A\(/
1076:     method.params = res unless method.params
1077: 
1078:     if method.block_params.nil? then
1079:       skip_tkspace false
1080:       read_documentation_modifiers method, RDoc::METHOD_MODIFIERS
1081:     end
1082:   end

parse_module(container, single, tk, comment) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1084
1084:   def parse_module(container, single, tk, comment)
1085:     container, name_t = get_class_or_module container
1086: 
1087:     name = name_t.name
1088: 
1089:     mod = container.add_module RDoc::NormalModule, name
1090:     mod.record_location @top_level
1091: 
1092:     read_documentation_modifiers mod, RDoc::CLASS_MODIFIERS
1093:     parse_statements(mod)
1094:     mod.comment = comment
1095: 
1096:     @stats.add_module mod
1097:   end

parse_require(context, comment) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1099
1099:   def parse_require(context, comment)
1100:     skip_tkspace_comment
1101:     tk = get_tk
1102: 
1103:     if TkLPAREN === tk then
1104:       skip_tkspace_comment
1105:       tk = get_tk
1106:     end
1107: 
1108:     name = tk.text if TkSTRING === tk
1109: 
1110:     if name then
1111:       context.add_require RDoc::Require.new(name, comment)
1112:     else
1113:       unget_tk tk
1114:     end
1115:   end

parse_statements(container, single = NORMAL, current_method = nil, comment = '') click to toggle source

The core of the ruby parser.

      # File lib/rdoc/parser/ruby.rb, line 1120
1120:   def parse_statements(container, single = NORMAL, current_method = nil,
1121:                        comment = '')
1122:     nest = 1
1123:     save_visibility = container.visibility
1124: 
1125:     non_comment_seen = true
1126: 
1127:     while tk = get_tk do
1128:       keep_comment = false
1129: 
1130:       non_comment_seen = true unless TkCOMMENT === tk
1131: 
1132:       case tk
1133:       when TkNL then
1134:         skip_tkspace
1135:         tk = get_tk
1136: 
1137:         if TkCOMMENT === tk then
1138:           if non_comment_seen then
1139:             # Look for RDoc in a comment about to be thrown away
1140:             parse_comment container, tk, comment unless comment.empty?
1141: 
1142:             comment = ''
1143:             non_comment_seen = false
1144:           end
1145: 
1146:           while TkCOMMENT === tk do
1147:             comment << tk.text << "\n"
1148: 
1149:             tk = get_tk        # this is the newline
1150:             skip_tkspace false # leading spaces
1151:             tk = get_tk
1152:           end
1153: 
1154:           unless comment.empty? then
1155:             look_for_directives_in container, comment
1156: 
1157:             if container.done_documenting then
1158:               container.ongoing_visibility = save_visibility
1159:             end
1160:           end
1161: 
1162:           keep_comment = true
1163:         else
1164:           non_comment_seen = true
1165:         end
1166: 
1167:         unget_tk tk
1168:         keep_comment = true
1169: 
1170:       when TkCLASS then
1171:         if container.document_children then
1172:           parse_class container, single, tk, comment
1173:         else
1174:           nest += 1
1175:         end
1176: 
1177:       when TkMODULE then
1178:         if container.document_children then
1179:           parse_module container, single, tk, comment
1180:         else
1181:           nest += 1
1182:         end
1183: 
1184:       when TkDEF then
1185:         if container.document_self then
1186:           parse_method container, single, tk, comment
1187:         else
1188:           nest += 1
1189:         end
1190: 
1191:       when TkCONSTANT then
1192:         if container.document_self then
1193:           parse_constant container, tk, comment
1194:         end
1195: 
1196:       when TkALIAS then
1197:         if container.document_self and not current_method then
1198:           parse_alias container, single, tk, comment
1199:         end
1200: 
1201:       when TkYIELD then
1202:         if current_method.nil? then
1203:           warn "Warning: yield outside of method" if container.document_self
1204:         else
1205:           parse_yield container, single, tk, current_method
1206:         end
1207: 
1208:       # Until and While can have a 'do', which shouldn't increase the nesting.
1209:       # We can't solve the general case, but we can handle most occurrences by
1210:       # ignoring a do at the end of a line.
1211: 
1212:       when  TkUNTIL, TkWHILE then
1213:         nest += 1
1214:         skip_optional_do_after_expression
1215: 
1216:       # 'for' is trickier
1217:       when TkFOR then
1218:         nest += 1
1219:         skip_for_variable
1220:         skip_optional_do_after_expression
1221: 
1222:       when TkCASE, TkDO, TkIF, TkUNLESS, TkBEGIN then
1223:         nest += 1
1224: 
1225:       when TkIDENTIFIER then
1226:         if nest == 1 and current_method.nil? then
1227:           case tk.name
1228:           when 'private', 'protected', 'public', 'private_class_method',
1229:                'public_class_method', 'module_function' then
1230:             parse_visibility container, single, tk
1231:             keep_comment = true
1232:           when 'attr' then
1233:             parse_attr container, single, tk, comment
1234:           when /^attr_(reader|writer|accessor)$/ then
1235:             parse_attr_accessor container, single, tk, comment
1236:           when 'alias_method' then
1237:             parse_alias container, single, tk, comment if
1238:               container.document_self
1239:           when 'require', 'include' then
1240:             # ignore
1241:           else
1242:             if container.document_self and comment =~ /\A#\#$/ then
1243:               case comment
1244:               when /^# +:?attr(_reader|_writer|_accessor)?:/ then
1245:                 parse_meta_attr container, single, tk, comment
1246:               else
1247:                 parse_meta_method container, single, tk, comment
1248:               end
1249:             end
1250:           end
1251:         end
1252: 
1253:         case tk.name
1254:         when "require" then
1255:           parse_require container, comment
1256:         when "include" then
1257:           parse_include container, comment
1258:         end
1259: 
1260:       when TkEND then
1261:         nest -= 1
1262:         if nest == 0 then
1263:           read_documentation_modifiers container, RDoc::CLASS_MODIFIERS
1264:           container.ongoing_visibility = save_visibility
1265: 
1266:           parse_comment container, tk, comment unless comment.empty?
1267: 
1268:           return
1269:         end
1270:       end
1271: 
1272:       comment = '' unless keep_comment
1273: 
1274:       begin
1275:         get_tkread
1276:         skip_tkspace false
1277:       end while peek_tk == TkNL
1278:     end
1279:   end

parse_symbol_arg(no = nil) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1281
1281:   def parse_symbol_arg(no = nil)
1282:     args = []
1283:     skip_tkspace_comment
1284:     case tk = get_tk
1285:     when TkLPAREN
1286:       loop do
1287:         skip_tkspace_comment
1288:         if tk1 = parse_symbol_in_arg
1289:           args.push tk1
1290:           break if no and args.size >= no
1291:         end
1292: 
1293:         skip_tkspace_comment
1294:         case tk2 = get_tk
1295:         when TkRPAREN
1296:           break
1297:         when TkCOMMA
1298:         else
1299:           warn("unexpected token: '#{tk2.inspect}'") if $DEBUG_RDOC
1300:           break
1301:         end
1302:       end
1303:     else
1304:       unget_tk tk
1305:       if tk = parse_symbol_in_arg
1306:         args.push tk
1307:         return args if no and args.size >= no
1308:       end
1309: 
1310:       loop do
1311:         skip_tkspace false
1312: 
1313:         tk1 = get_tk
1314:         unless TkCOMMA === tk1 then
1315:           unget_tk tk1
1316:           break
1317:         end
1318: 
1319:         skip_tkspace_comment
1320:         if tk = parse_symbol_in_arg
1321:           args.push tk
1322:           break if no and args.size >= no
1323:         end
1324:       end
1325:     end
1326:     args
1327:   end

parse_symbol_in_arg() click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1329
1329:   def parse_symbol_in_arg
1330:     case tk = get_tk
1331:     when TkSYMBOL
1332:       tk.text.sub(/^:/, '')
1333:     when TkSTRING
1334:       eval @read[1]
1335:     else
1336:       warn("Expected symbol or string, got #{tk.inspect}") if $DEBUG_RDOC
1337:       nil
1338:     end
1339:   end

parse_top_level_statements(container) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1341
1341:   def parse_top_level_statements(container)
1342:     comment = collect_first_comment
1343:     look_for_directives_in(container, comment)
1344:     container.comment = comment unless comment.empty?
1345:     parse_statements container, NORMAL, nil, comment
1346:   end

parse_visibility(container, single, tk) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1348
1348:   def parse_visibility(container, single, tk)
1349:     singleton = (single == SINGLE)
1350: 
1351:     vis_type = tk.name
1352: 
1353:     vis = case vis_type
1354:           when 'private'   then :private
1355:           when 'protected' then :protected
1356:           when 'public'    then :public
1357:           when 'private_class_method' then
1358:             singleton = true
1359:             :private
1360:           when 'public_class_method' then
1361:             singleton = true
1362:             :public
1363:           when 'module_function' then
1364:             singleton = true
1365:             :public
1366:           else
1367:             raise RDoc::Error, "Invalid visibility: #{tk.name}"
1368:           end
1369: 
1370:     skip_tkspace_comment false
1371: 
1372:     case peek_tk
1373:       # Ryan Davis suggested the extension to ignore modifiers, because he
1374:       # often writes
1375:       #
1376:       #   protected unless $TESTING
1377:       #
1378:     when TkNL, TkUNLESS_MOD, TkIF_MOD, TkSEMICOLON then
1379:       container.ongoing_visibility = vis
1380:     else
1381:       if vis_type == 'module_function' then
1382:         args = parse_symbol_arg
1383:         container.set_visibility_for args, :private, false
1384: 
1385:         module_functions = []
1386: 
1387:         container.methods_matching args do |m|
1388:           s_m = m.dup
1389:           s_m.singleton = true if RDoc::AnyMethod === s_m
1390:           s_m.visibility = :public
1391:           module_functions << s_m
1392:         end
1393: 
1394:         module_functions.each do |s_m|
1395:           case s_m
1396:           when RDoc::AnyMethod then
1397:             container.add_method s_m
1398:           when RDoc::Attr then
1399:             container.add_attribute s_m
1400:           end
1401:         end
1402:       else
1403:         args = parse_symbol_arg
1404:         container.set_visibility_for args, vis, singleton
1405:       end
1406:     end
1407:   end

parse_yield(context, single, tk, method) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1409
1409:   def parse_yield(context, single, tk, method)
1410:     return if method.block_params
1411: 
1412:     get_tkread
1413:     @scanner.instance_eval { @continue = false }
1414:     method.block_params = parse_method_or_yield_parameters
1415:   end

read_directive(allowed) click to toggle source

Directives are modifier comments that can appear after class, module, or method names. For example:

  def fred # :yields: a, b

or:

  class MyClass # :nodoc:

We return the directive name and any parameters as a two element array

      # File lib/rdoc/parser/ruby.rb, line 1429
1429:   def read_directive(allowed)
1430:     tk = get_tk
1431:     result = nil
1432: 
1433:     if TkCOMMENT === tk then
1434:       if tk.text =~ /\s*:?(\w+):\s*(.*)/ then
1435:         directive = $1.downcase
1436:         if allowed.include? directive then
1437:           result = [directive, $2]
1438:         end
1439:       end
1440:     else
1441:       unget_tk tk
1442:     end
1443: 
1444:     result
1445:   end

read_documentation_modifiers(context, allow) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1447
1447:   def read_documentation_modifiers(context, allow)
1448:     dir = read_directive(allow)
1449: 
1450:     case dir[0]
1451:     when "notnew", "not_new", "not-new" then
1452:       context.dont_rename_initialize = true
1453: 
1454:     when "nodoc" then
1455:       context.document_self = false
1456:       if dir[1].downcase == "all"
1457:         context.document_children = false
1458:       end
1459: 
1460:     when "doc" then
1461:       context.document_self = true
1462:       context.force_documentation = true
1463: 
1464:     when "yield", "yields" then
1465:       unless context.params.nil?
1466:         context.params.sub!(/(,|)\s*&\w+/,'') # remove parameter &proc
1467:       end
1468: 
1469:       context.block_params = dir[1]
1470: 
1471:     when "arg", "args" then
1472:       context.params = dir[1]
1473:     end if dir
1474:   end

remove_private_comments(comment) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1476
1476:   def remove_private_comments(comment)
1477:     comment.gsub!(/^#--\n.*?^#\+\+/, '')
1478:     comment.sub!(/^#--\n.*/, '')
1479:   end

scan() click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1481
1481:   def scan
1482:     reset
1483: 
1484:     catch :eof do
1485:       catch :enddoc do
1486:         begin
1487:           parse_top_level_statements @top_level
1488:         rescue StandardError => e
1489:           bytes = ''
1490: 
1491:           20.times do @scanner.ungetc end
1492:           count = 0
1493:           60.times do |i|
1494:             count = i
1495:             byte = @scanner.getc
1496:             break unless byte
1497:             bytes << byte
1498:           end
1499:           count -= 20
1500:           count.times do @scanner.ungetc end
1501: 
1502:           $stderr.puts #{self.class} failure around line #{@scanner.line_no} of#{@file_name}
1503: 
1504:           unless bytes.empty? then
1505:             $stderr.puts
1506:             $stderr.puts bytes.inspect
1507:           end
1508: 
1509:           raise e
1510:         end
1511:       end
1512:     end
1513: 
1514:     @top_level
1515:   end

skip_for_variable() click to toggle source

skip the var [in] part of a ‘for’ statement

      # File lib/rdoc/parser/ruby.rb, line 1572
1572:   def skip_for_variable
1573:     skip_tkspace false
1574:     tk = get_tk
1575:     skip_tkspace false
1576:     tk = get_tk
1577:     unget_tk(tk) unless TkIN === tk
1578:   end

skip_method(container) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1580
1580:   def skip_method container
1581:     meth = RDoc::AnyMethod.new "", "anon"
1582:     parse_method_parameters meth
1583:     parse_statements container, false, meth
1584:   end

skip_optional_do_after_expression() click to toggle source

while, until, and for have an optional do

      # File lib/rdoc/parser/ruby.rb, line 1525
1525:   def skip_optional_do_after_expression
1526:     skip_tkspace false
1527:     tk = get_tk
1528:     case tk
1529:     when TkLPAREN, TkfLPAREN then
1530:       end_token = TkRPAREN
1531:     else
1532:       end_token = TkNL
1533:     end
1534: 
1535:     b_nest = 0
1536:     nest = 0
1537:     @scanner.instance_eval { @continue = false }
1538: 
1539:     loop do
1540:       case tk
1541:       when TkSEMICOLON then
1542:         break if b_nest.zero?
1543:       when TkLPAREN, TkfLPAREN then
1544:         nest += 1
1545:       when TkBEGIN then
1546:         b_nest += 1
1547:       when TkEND then
1548:         b_nest -= 1
1549:       when TkDO
1550:         break if nest.zero?
1551:       when end_token then
1552:         if end_token == TkRPAREN
1553:           nest -= 1
1554:           break if @scanner.lex_state == EXPR_END and nest.zero?
1555:         else
1556:           break unless @scanner.continue
1557:         end
1558:       when nil then
1559:         break
1560:       end
1561:       tk = get_tk
1562:     end
1563: 
1564:     skip_tkspace false
1565: 
1566:     get_tk if TkDO === peek_tk
1567:   end

skip_tkspace_comment(skip_nl = true) click to toggle source

Skip spaces until a comment is found

      # File lib/rdoc/parser/ruby.rb, line 1589
1589:   def skip_tkspace_comment(skip_nl = true)
1590:     loop do
1591:       skip_tkspace skip_nl
1592:       return unless TkCOMMENT === peek_tk
1593:       get_tk
1594:     end
1595:   end

warn(msg) click to toggle source

      # File lib/rdoc/parser/ruby.rb, line 1597
1597:   def warn(msg)
1598:     return if @options.quiet
1599:     msg = make_message msg
1600:     $stderr.puts msg
1601:   end

Home Classes Methods

In Files

Parent

Methods

Included Modules

Files

Class Index