Ripper is a Ruby script parser.
You can get information from the parser with event-based style. Information such as abstract syntax trees or simple lexical analysis of the Ruby program.
Ripper provides an easy interface for parsing your program into a symbolic expression tree (or S-expression).
Understanding the output of the parser may come as a challenge, it's recommended you use PP to format the output for legibility.
require 'ripper' require 'pp' pp Ripper.sexp('def hello(world) "Hello, #{world}!"; end') #=> [:program, [[:def, [:@ident, "hello", [1, 4]], [:paren, [:params, [[:@ident, "world", [1, 10]]], nil, nil, nil, nil, nil, nil]], [:bodystmt, [[:string_literal, [:string_content, [:@tstring_content, "Hello, ", [1, 18]], [:string_embexpr, [[:var_ref, [:@ident, "world", [1, 27]]]]], [:@tstring_content, "!", [1, 33]]]]], nil, nil, nil]]]]
You can see in the example above, the expression starts with :program
.
From here, a method definition at :def
, followed by the method's identifier :@ident
. After the method's identifier comes the parentheses :paren
and the method parameters under :params
.
Next is the method body, starting at :bodystmt
(stmt
meaning statement), which contains the full definition of the method.
In our case, we're simply returning a String, so next we have the :string_literal
expression.
Within our :string_literal
you'll notice two @tstring_content
, this is the literal part for Hello,
and !
. Between the two @tstring_content
statements is a :string_embexpr
, where embexpr is an embedded expression. Our expression consists of a local variable, or var_ref
, with the identifier (@ident
) of world
.
ruby 1.9 (support CVS HEAD only)
bison 1.28 or later (Other yaccs do not work)
Ruby License.
Minero Aoki
This array contains name of all ripper events.
This array contains name of parser events.
This array contains name of scanner events.
version of Ripper
static VALUE parser_dedent_string(VALUE self, VALUE input, VALUE width) { int wid, col; StringValue(input); wid = NUM2UINT(width); col = dedent_string(input, wid); return INT2NUM(col); }
Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token, state]
.
require 'ripper' require 'pp' pp Ripper.lex("def m(a) nil end") #=> [[[1, 0], :on_kw, "def", Ripper::EXPR_FNAME ], [[1, 3], :on_sp, " ", Ripper::EXPR_FNAME ], [[1, 4], :on_ident, "m", Ripper::EXPR_ENDFN ], [[1, 5], :on_lparen, "(", Ripper::EXPR_LABEL | Ripper::EXPR_BEG], [[1, 6], :on_ident, "a", Ripper::EXPR_ARG ], [[1, 7], :on_rparen, ")", Ripper::EXPR_ENDFN ], [[1, 8], :on_sp, " ", Ripper::EXPR_BEG ], [[1, 9], :on_kw, "nil", Ripper::EXPR_END ], [[1, 12], :on_sp, " ", Ripper::EXPR_END ], [[1, 13], :on_kw, "end", Ripper::EXPR_END ]]
# File ext/ripper/lib/ripper/lexer.rb, line 43 def Ripper.lex(src, filename = '-', lineno = 1) Lexer.new(src, filename, lineno).lex end
static VALUE ripper_lex_state_name(VALUE self, VALUE state) { return rb_parser_lex_state_name(NUM2INT(state)); }
Create a new Ripper object. src must be a String, an IO, or an Object which has gets method.
This method does not starts parsing. See also #parse and ::parse.
static VALUE ripper_initialize(int argc, VALUE *argv, VALUE self) { struct parser_params *parser; VALUE src, fname, lineno; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); rb_scan_args(argc, argv, "12", &src, &fname, &lineno); if (RB_TYPE_P(src, T_FILE)) { lex_gets = ripper_lex_io_get; } else if (rb_respond_to(src, id_gets)) { lex_gets = ripper_lex_get_generic; } else { StringValue(src); lex_gets = lex_get_str; } lex_input = src; parser->eofp = 0; if (NIL_P(fname)) { fname = STR_NEW2("(ripper)"); OBJ_FREEZE(fname); } else { StringValueCStr(fname); fname = rb_str_new_frozen(fname); } parser_initialize(parser); ruby_sourcefile_string = fname; ruby_sourcefile = RSTRING_PTR(fname); ruby_sourceline = NIL_P(lineno) ? 0 : NUM2INT(lineno) - 1; return Qnil; }
Parses src
and create S-exp tree. Returns more readable tree rather than ::sexp_raw. This method is mainly for developer use.
require 'ripper' require 'pp' pp Ripper.sexp("def m(a) nil end") #=> [:program, [[:def, [:@ident, "m", [1, 4]], [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil]], [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
# File ext/ripper/lib/ripper/sexp.rb, line 30 def Ripper.sexp(src, filename = '-', lineno = 1) builder = SexpBuilderPP.new(src, filename, lineno) sexp = builder.parse sexp unless builder.error? end
Parses src
and create S-exp tree. This method is mainly for developer use.
require 'ripper' require 'pp' pp Ripper.sexp_raw("def m(a) nil end") #=> [:program, [:stmts_add, [:stmts_new], [:def, [:@ident, "m", [1, 4]], [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]], [:bodystmt, [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
# File ext/ripper/lib/ripper/sexp.rb, line 56 def Ripper.sexp_raw(src, filename = '-', lineno = 1) builder = SexpBuilder.new(src, filename, lineno) sexp = builder.parse sexp unless builder.error? end
Parses src
and return a string which was matched to pattern
. pattern
should be described as Regexp.
require 'ripper' p Ripper.slice('def m(a) nil end', 'ident') #=> "m" p Ripper.slice('def m(a) nil end', '[ident lparen rparen]+') #=> "m(a)" p Ripper.slice("<<EOS\nstring\nEOS", 'heredoc_beg nl $(tstring_content*) heredoc_end', 1) #=> "string\n"
# File ext/ripper/lib/ripper/lexer.rb, line 142 def Ripper.slice(src, pattern, n = 0) if m = token_match(src, pattern) then m.string(n) else nil end end
Tokenizes the Ruby program and returns an array of strings.
p Ripper.tokenize("def m(a) nil end") # => ["def", " ", "m", "(", "a", ")", " ", "nil", " ", "end"]
# File ext/ripper/lib/ripper/lexer.rb, line 20 def Ripper.tokenize(src, filename = '-', lineno = 1) Lexer.new(src, filename, lineno).tokenize end
Return column number of current parsing line. This number starts from 0.
static VALUE ripper_column(VALUE self) { struct parser_params *parser; long col; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (NIL_P(parser->parsing_thread)) return Qnil; col = parser->tokp - lex_pbeg; return LONG2NUM(col); }
Return encoding of the source.
VALUE rb_parser_encoding(VALUE vparser) { struct parser_params *parser; TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser); return rb_enc_from_encoding(current_enc); }
Return true if parsed source ended by +_END_+.
VALUE rb_parser_end_seen_p(VALUE vparser) { struct parser_params *parser; TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser); return ruby__end__seen ? Qtrue : Qfalse; }
Return true if parsed source has errors.
static VALUE ripper_error_p(VALUE vparser) { struct parser_params *parser; TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser); return parser->error_p ? Qtrue : Qfalse; }
Return current parsing filename.
static VALUE ripper_filename(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } return ruby_sourcefile_string; }
Return line number of current parsing line. This number starts from 1.
static VALUE ripper_lineno(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (NIL_P(parser->parsing_thread)) return Qnil; return INT2NUM(ruby_sourceline); }
Start parsing and returns the value of the root action.
static VALUE ripper_parse(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (!NIL_P(parser->parsing_thread)) { if (parser->parsing_thread == rb_thread_current()) rb_raise(rb_eArgError, "Ripper#parse is not reentrant"); else rb_raise(rb_eArgError, "Ripper#parse is not multithread-safe"); } parser->parsing_thread = rb_thread_current(); rb_ensure(ripper_parse0, self, ripper_ensure, self); return parser->result; }
Return scanner state of current token.
static VALUE ripper_state(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (NIL_P(parser->parsing_thread)) return Qnil; return INT2NUM(lex_state); }
Get yydebug.
VALUE rb_parser_get_yydebug(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); return yydebug ? Qtrue : Qfalse; }
Set yydebug.
VALUE rb_parser_set_yydebug(VALUE self, VALUE flag) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); yydebug = RTEST(flag); return flag; }
# File ext/ripper/lib/ripper/core.rb, line 32 def _dispatch_0() nil end
# File ext/ripper/lib/ripper/core.rb, line 33 def _dispatch_1(a) a end
# File ext/ripper/lib/ripper/core.rb, line 34 def _dispatch_2(a, b) a end
# File ext/ripper/lib/ripper/core.rb, line 35 def _dispatch_3(a, b, c) a end
# File ext/ripper/lib/ripper/core.rb, line 36 def _dispatch_4(a, b, c, d) a end
# File ext/ripper/lib/ripper/core.rb, line 37 def _dispatch_5(a, b, c, d, e) a end
# File ext/ripper/lib/ripper/core.rb, line 38 def _dispatch_6(a, b, c, d, e, f) a end
# File ext/ripper/lib/ripper/core.rb, line 39 def _dispatch_7(a, b, c, d, e, f, g) a end
This method is called when the parser found syntax error.
# File ext/ripper/lib/ripper/core.rb, line 60 def compile_error(msg) end
static VALUE parser_dedent_string(VALUE self, VALUE input, VALUE width) { int wid, col; StringValue(input); wid = NUM2UINT(width); col = dedent_string(input, wid); return INT2NUM(col); }
This method is called when weak warning is produced by the parser. fmt
and args
is printf style.
# File ext/ripper/lib/ripper/core.rb, line 51 def warn(fmt, *args) end
This method is called when strong warning is produced by the parser. fmt
and args
is printf style.
# File ext/ripper/lib/ripper/core.rb, line 56 def warning(fmt, *args) end
Ruby Core © 1993–2017 Yukihiro Matsumoto
Licensed under the Ruby License.
Ruby Standard Library © contributors
Licensed under their own licenses.