summaryrefslogtreecommitdiffstats
path: root/include/ruby/encoding.h
Commit message (Collapse)AuthorAgeFilesLines
* * include/ruby/oniguruma.h (OnigEncoding): removed auxiliary_data.nobu2008-07-041-9/+10
| | | | | | | | | | | | | * include/ruby/encoding.h (ENC_DUMMY_P): moved dummy encoding flag to rb_encoding from Encoding instance. * encoding.c (rb_encoding_list): list of Encoding instances. * encoding.c (struct rb_encoding_entry): moved base encoding from instance variable. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@17875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_utf8_encindex): defined.naruse2008-06-291-1/+2
| | | | | | * include/ruby/encoding.h (rb_utf8_encindex): ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@17724 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_filesystem_encoding): defined.naruse2008-06-161-0/+1
| | | | | | * include/ruby/encoding.h (rb_filesystem_encoding): added. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@17372 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h: not to use varargs.h since requiring C89.nobu2008-06-101-5/+1
| | | | git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@17066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c, include/ruby/encoding.hnaruse2008-05-201-2/+2
| | | | | | | (rb_enc_associate, rb_enc_associate_index): returns obj. [ruby-dev:34778] git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@16499 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_ascii8bit_encoding): use ENCINDEX_ASCII.naruse2008-05-201-0/+1
| | | | | | | | | * encoding.c, include/ruby/encoding.h (rb_ascii8bit_encindex): added. * encoding.c (rb_locale_encoding): use rb_usascii_encoding(). git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@16498 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c, include/ruby/encodng.h (rb_str_transcode):naruse2008-05-201-0/+2
| | | | | | | | | | C API of encoding conversion for Ruby object. VALUE rb_str_transcode(VALUE str, VALUE to). * transcode.c (str_encode, str_encode_bang): rename from rb_tr_transcode or rb_str_transcode_bang. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@16496 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * regparse.c (PINC): use optimized enclen() instead ofmatz2008-05-191-6/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ONIGENC_MBC_ENC_LEN(). * regparse.c (PFETCH): ditto. * regparse.c (PFETCH): small optimization. * regexec.c (slow_search): single byte encoding optimization. * regenc.h (enclen): avoid calling function when encoding's min_len == max_len. * re.c (rb_reg_regsub): rb_enc_ascget() optimization for single byte encoding. * re.c (rb_reg_search): avoid allocating new re_registers if we already have MatchData. * re.c (match_init_copy): avoid unnecessary onig_region_free() before onig_region_copy. * encoding.c (rb_enc_get_index): remove implicit enc_capable check each time. * encoding.c (rb_enc_set_index): ditto. * encoding.c (enc_compatible_p): small refactoring. * include/ruby/encoding.h (rb_enc_dummy_p): inline rb_enc_dummy_p() and export related code. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@16477 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * re.c (rb_memsearch_ss): simple shift search.naruse2008-03-171-0/+1
| | | | | | | | | | | | | | | | | | | | * re.c (rb_memsearch_qs): quick search. * re.c (rb_memsearch_qs_utf8): quick search for UTF-8 string. * re.c (rb_memsearch_qs_utf8_hash): hash functions for above. * re.c (rb_memsearch): use above functions. * string.c (rb_str_index): give enc to rb_memsearch. * include/ruby/intern.h (rb_memsearch): move to encoding.h. * include/ruby/encoding.h (rb_memsearch): move from intern.h. * common.mk (PREP): add dependency. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15792 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_str_coderange_scan_restartable): addednobu2008-02-281-0/+1
| | | | | | | | | | | | prototype. * string.c (rb_str_coderange_scan_restartable, rb_str_times): removed unsed variables. * string.c (rb_enc_str_copy): unused now. may be used in future? git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15620 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_enc_asciicompat): dummy encoding is notnaruse2008-02-211-1/+1
| | | | | | ascii compatible. [ruby-dev:33878] git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15559 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (ENC_CODERANGE_AND): fix broken case. [ruby-dev:33826]naruse2008-02-171-7/+6
| | | | | | * string.c (rb_str_times): fix broken case. [ruby-dev:33826] git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15525 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (ENC_CODERANGE_AND): added.naruse2008-02-171-0/+7
| | | | | | | | * string.c (rb_str_plus, srb_str_times): keep coderange. * parse.y (STR_NEW0) use rb_usascii_str_new. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15519 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/oniguruma.h: precise mbclen API redesigned to avoidakr2008-01-271-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | inline functions. (onigenc_mbclen_charfound): removed. (onigenc_mbclen_needmore): removed. (onigenc_mbclen_recover): removed. (ONIGENC_MBCLEN_CHARFOUND): removed. (ONIGENC_MBCLEN_CHARFOUND_P): defined. (ONIGENC_MBCLEN_CHARFOUND_LEN): defined. (ONIGENC_MBCLEN_INVALID): removed. (ONIGENC_MBCLEN_INVALID_P): defined. (ONIGENC_MBCLEN_NEEDMORE): removed. (ONIGENC_MBCLEN_NEEDMORE_P): defined. (ONIGENC_MBCLEN_NEEDMORE_LEN): defined. (ONIGENC_MBC_ENC_LEN): use onigenc_mbclen_approximate. * regenc.c (onigenc_mbclen_approximate): defined. * include/ruby/encoding.h (MBCLEN_CHARFOUND): removed. (MBCLEN_INVALID): removed. (MBCLEN_NEEDMORE): removed. (MBCLEN_CHARFOUND_P): defined. (MBCLEN_INVALID_P): defined. (MBCLEN_NEEDMORE_P): defined. (MBCLEN_CHARFOUND_LEN): defined. (MBCLEN_NEEDMORE_LEN): defined. * encoding.c: use new API. * re.c: ditto. * string.c: ditto. * parse.y: ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15280 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_usascii_encindex): added prototype.nobu2008-01-261-0/+1
| | | | | | | | * include/ruby/intern.h (rb_usascii_str_new, rb_usascii_str_new2): ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15248 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_enc_is_newline): parenthesized arguments.nobu2008-01-241-1/+1
| | | | git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * string.c (rb_str_each_line): use memchr(3) for faster newlinematz2008-01-231-1/+1
| | | | | | | | | | search. * io.c (appendline): remove unused arguments * io.c (rb_io_getline_fast): make much simpler (and faster). git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.{c, h} (rb_usascii_encoding): added.naruse2008-01-231-0/+1
| | | | git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15194 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * parse.y (rb_intern3): do not call rb_enc_mbclen() if *m ismatz2008-01-211-0/+3
| | | | | | | | | ASCII. [ruby-talk:287225] * string.c (rb_str_each_line): use rb_enc_is_newline() to gain performance if the record separator ($/) is not modified. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15163 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * file.c (sys_fail2): get rid of unlimited alloca.nobu2008-01-161-0/+2
| | | | | | | * io.c (mode_enc, pipe_open, rb_io_s_popen): ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@15073 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/oniguruma.h (OnigEncodingType): new memberakr2008-01-071-1/+1
| | | | | | | | | | | | | ruby_encoding_index to avoid linear search in rb_enc_to_index. * include/ruby/encoding.h (rb_enc_to_index): macro defined to use ruby_encoding_index. * encoding.c (rb_enc_to_index): removed. (enc_register_at): initialize ruby_encoding_index member. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14931 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_internal_get_index): extracted fromakr2008-01-071-3/+27
| | | | | | | | | | | | | | | | | | | | | rb_enc_get_index. (rb_enc_internal_set_index): extracted from rb_enc_associate_index * include/ruby/encoding.h (ENCODING_SET): work over ENCODING_INLINE_MAX. (ENCODING_GET): ditto. (ENCODING_IS_ASCII8BIT): defined. (ENCODING_CODERANGE_SET): defined. * re.c (rb_reg_fixed_encoding_p): use ENCODING_IS_ASCII8BIT. * string.c (rb_enc_str_buf_cat): use ENCODING_IS_ASCII8BIT. * parse.y (reg_fragment_setenc_gen): use ENCODING_IS_ASCII8BIT. * marshal.c (has_ivars): use ENCODING_IS_ASCII8BIT. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14922 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * $Date$ keyword removed to avoid inclusion of locale dependentakr2008-01-061-1/+0
| | | | | | | string. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_enc_str_buf_cat): declared.akr2008-01-061-0/+1
| | | | | | | | | | | | | | | | * string.c (coderange_scan): extracted from rb_enc_str_coderange. (rb_enc_str_coderange): use coderange_scan. (rb_str_shared_replace): copy encoding and coderange. (rb_enc_str_buf_cat): new function for linear complexity string accumulation with encoding. (rb_str_sub_bang): don't conflict substituted part and replacement. (str_gsub): use rb_enc_str_buf_cat. (rb_str_clear): clear coderange. * re.c (rb_reg_regsub): use rb_enc_str_buf_cat. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14910 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/intern.h, re.c (rb_reg_new): keep interface same asnobu2008-01-041-1/+2
| | | | | | | | | | | | | | | | 1.8. [ruby-core:14583] * include/ruby/intern.h, re.c (rb_reg_new_str): renamed, and defines HAVE_RB_REG_NEW_STR macro to tell if it is available. * include/ruby/encoding.h (rb_enc_reg_new): added. * insns.def (toregexp), marshal.c (r_object0): use rb_reg_new_str(). * re.c (rb_reg_regcomp, rb_reg_s_union): ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14884 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_isalnum): defined.akr2008-01-031-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (rb_isalpha): ditto. (rb_isblank): ditto. (rb_iscntrl): ditto. (rb_isdigit): ditto. (rb_isgraph): ditto. (rb_islower): ditto. (rb_isprint): ditto. (rb_ispunct): ditto. (rb_isspace): ditto. (rb_isupper): ditto. (rb_isxdigit): ditto. (rb_tolower): ditto. (rb_toupper): ditto. * include/ruby/ruby.h: don't include include/ruby/encoding.h. (rb_isascii): defined. (rb_isalnum): declared. (rb_isalpha): ditto. (rb_isblank): ditto. (rb_iscntrl): ditto. (rb_isdigit): ditto. (rb_isgraph): ditto. (rb_islower): ditto. (rb_isprint): ditto. (rb_ispunct): ditto. (rb_isspace): ditto. (rb_isupper): ditto. (rb_isxdigit): ditto. (rb_tolower): ditto. (rb_toupper): ditto. (ISASCII): simplified. (ISPRINT): ditto. (ISSPACE): ditto. (ISUPPER): ditto. (ISLOWER): ditto. (ISALNUM): ditto. (ISALPHA): ditto. (ISDIGIT): ditto. (ISXDIGIT): ditto. (TOUPPER): ditto. (TOLOWER): ditto. * include/ruby/encoding.h (rb_isascii): removed. (rb_isalnum): ditto. (rb_isalpha): ditto. (rb_isblank): ditto. (rb_iscntrl): ditto. (rb_isdigit): ditto. (rb_isgraph): ditto. (rb_islower): ditto. (rb_isprint): ditto. (rb_ispunct): ditto. (rb_isspace): ditto. (rb_isupper): ditto. (rb_isxdigit): ditto. (rb_tolower): ditto. (rb_toupper): ditto. * common.mk: dependency updated. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14865 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_isascii): simplified.akr2008-01-031-13/+13
| | | | | | | | | | | | | | | | | | | | | (rb_isalnum): call onigenc_ascii_is_code_ctype without indirect call. (rb_isalpha): ditto. (rb_isblank): ditto. (rb_iscntrl): ditto. (rb_isdigit): ditto. (rb_isgraph): ditto. (rb_islower): ditto. (rb_isprint): ditto. (rb_ispunct): ditto. (rb_isspace): ditto. (rb_isupper): ditto. (rb_isxdigit): ditto. * include/ruby/oniguruma.h (onigenc_ascii_is_code_ctype): declaration moved from regenc.h. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14864 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_isascii): defined.akr2008-01-011-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (rb_isalnum): ditto. (rb_isalpha): ditto. (rb_isblank): ditto. (rb_iscntrl): ditto. (rb_isdigit): ditto. (rb_isgraph): ditto. (rb_islower): ditto. (rb_isprint): ditto. (rb_ispunct): ditto. (rb_isspace): ditto. (rb_isupper): ditto. (rb_isxdigit): ditto. (rb_tolower): ditto. (rb_toupper): ditto. * include/ruby/st.h (st_strcasecmp): declared. (st_strncasecmp): ditto. * st.c (type_strcasehash): use st_strcasecmp instead of strcasecmp. (st_strcasecmp): defined. (st_strncasecmp): ditto. * include/ruby/ruby.h: include include/ruby/encoding.h. (ISASCII): use rb_isascii. (ISPRINT): use rb_isprint. (ISSPACE): use rb_isspace. (ISUPPER): use rb_isupper. (ISLOWER): use rb_islower. (ISALNUM): use rb_isalnum. (ISALPHA): use rb_isalpha. (ISDIGIT): use rb_isdigit. (ISXDIGIT): use rb_isxdigit. (TOUPPER): defined. (TOLOWER): ditto. (STRCASECMP): ditto. (STRNCASECMP): ditto. * dir.c, encoding.c, file.c, hash.c, process.c, ruby.c, time.c, transcode.c, ext/readline/readline.c: use locale insensitive functions. [ruby-core:14662] git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14829 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_enc_sprintf, rb_enc_vsprintf): prototyped.nobu2007-12-301-0/+7
| | | | | | | | * sprintf.c (rb_enc_sprintf, rb_enc_vsprintf): new functions to format arguments with encoding. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14806 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * ext/strscan/strscan.c (str_new): new function for allocate an stringakr2007-12-281-1/+1
| | | | | | | | | | | | with encoding propagation. (extract_range): use str_new. (extract_beg_len): ditto. (strscan_peek): ditto. (strscan_rest): ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14772 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_locale_encoding): defined.akr2007-12-281-0/+1
| | | | | | | | * include/ruby/encoding.h (rb_locale_encoding): declared. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14768 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_codelen): show codepoint in error message.akr2007-12-271-1/+1
| | | | | | | | | | * include/ruby/encoding.h (rb_enc_codelen): comment it returns positive integer. * string.c (rb_str_concat): rb_enc_codelen doesn't return 0. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14733 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.h (rb_enc_mbc_to_codepoint): wrapper fornobu2007-12-251-4/+7
| | | | | | | | | ONIGENC_MBC_TO_CODE(). * string.c (rb_str_succ): deal with invalid sequence as binary. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14692 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_enc_left_char_head): new utility macro.matz2007-12-241-0/+3
| | | | | | | | | | | | | * include/ruby/encoding.h (rb_enc_right_char_head): ditto. * io.c (appendline): does multibyte RS search in the function. * io.c (prepare_getline_args): RS may be nil. * io.c (rb_io_getc): should process character based on external encoding, when transcoding required. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14619 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* comment updated.akr2007-12-231-4/+4
| | | | git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14526 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_codepoint): implemented to raise invalidakr2007-12-231-6/+6
| | | | | | | | | | encoding. * include/ruby/encoding.h (rb_enc_codepoint): macro is replaced as a declaration. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14524 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h, encoding.c, re.c, io.c, parse.y, numeric.c,akr2007-12-221-1/+1
| | | | | | | | | ruby.c, transcode.c: rename rb_ascii_encoding. to rb_ascii8bit_encoding. rb_ascii_encoding is ambiguous with ASCII-8BIT and US-ASCII. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14504 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_ascii_encoding): renamed from previousmatz2007-12-211-1/+1
| | | | | | rb_default_encoding(). git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14443 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_replicate): now creates first class encoding.nobu2007-12-211-7/+9
| | | | | | | | | | | | | | | | * encoding.c (rb_define_dummy_encoding): always based on the default encoding. * encoding.c (rb_enc_dummy_p): check if dummy. * encoding.c (enc_inspect): shows if dummy. * encoding.c (Init_Encoding): added dummy? method * include/ruby/encoding.h (ENCODING_INLINE_MAX): increased. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_init): use enc_register_at() directly.nobu2007-12-211-0/+1
| | | | | | | | | | | * encoding.c (rb_utf8_encoding): returns utf-8 encoding. * include/ruby/encoding.h (rb_utf8_encoding): prototyped. * parse.y (UTF8_ENC): uses rb_utf8_encoding(). git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14410 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c: include locale.hakr2007-12-211-0/+1
| | | | | | | | | | | | | | (rb_locale_charmap): new method Encoding.locale_charmap for nl_langinfo(CODESET). * include/ruby/encoding.h (rb_locale_charmap): declared. * main.c (main): call setlocale with LC_CTYPE. * ruby.c (locale_encoding): use rb_locale_charmap. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14380 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (ENC_CODERANGE_VALID): rename fromakr2007-12-171-1/+1
| | | | | | | | | ENC_CODERANGE_8BIT. * string.c (rb_enc_str_coderange): follow the renaming. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14257 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_ascget): renamed from rb_enc_get_ascii.akr2007-12-111-1/+1
| | | | | | | | | | * include/ruby/encoding.h: follow the renaming. * re.c: ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14195 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_get_ascii): add an argument to provide the akr2007-12-111-4/+4
| | | | | | | | | | | | | length of the returned character. * include/ruby/encoding.h (rb_enc_get_ascii): add the argument. * re.c (rb_reg_expr_str): modify rb_enc_get_ascii call. (rb_reg_quote): ditto. (rb_reg_regsub): ditto. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14190 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (str_transcode): allow non-registered encodings.nobu2007-12-101-0/+1
| | | | | | | [ruby-dev:32520] git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14182 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_mbclen): make it never fail.akr2007-12-081-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (rb_enc_nth): don't check the return value of rb_enc_mbclen. (rb_enc_strlen): ditto. (rb_enc_precise_mbclen): return needmore(1) if e <= p. (rb_enc_get_ascii): new function for extracting ASCII character. * include/ruby/encoding.h (rb_enc_get_ascii): declared. * include/ruby/regex.h (ismbchar): removed. * re.c (rb_reg_expr_str): use rb_enc_get_ascii. (unescape_escaped_nonascii): use rb_enc_precise_mbclen to determine the termination of escaped non-ASCII character. (unescape_nonascii): use rb_enc_precise_mbclen. (rb_reg_quote): use rb_enc_get_ascii. (rb_reg_regsub): use rb_enc_get_ascii. * string.c (rb_str_reverse) don't check the return value of rb_enc_mbclen. (rb_str_split_m): don't call rb_enc_mbclen with e <= p. * parse.y (is_identchar): use ISASCII. (parser_ismbchar): removed. (parser_precise_mbclen): new macro. (parser_isascii): new macro. (parser_tokadd_mbchar): use parser_precise_mbclen to check invalid character precisely. (parser_tokadd_string): use parser_isascii. (parser_yylex): ditto. (is_special_global_name): don't call is_identchar with e <= p. (rb_enc_symname_p): ditto. [ruby-dev:32455] * ext/tk/sample/tkextlib/vu/canvSticker2.rb: remove coding cookie because the encoding is not UTF-8. [ruby-dev:32475] git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c (rb_enc_precise_mbclen): new function for mbclen withakr2007-12-061-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | validation. * include/ruby/encoding.h (rb_enc_precise_mbclen): declared. (MBCLEN_CHARFOUND): new macro. (MBCLEN_INVALID): new macro. (MBCLEN_NEEDMORE): new macro. * include/ruby/oniguruma.h (OnigEncodingTypeST): replace mbc_enc_len by precise_mbc_enc_len. (ONIGENC_PRECISE_MBC_ENC_LEN): new macro. (ONIGENC_CONSTRUCT_MBCLEN_CHARFOUND): new macro. (ONIGENC_CONSTRUCT_MBCLEN_INVALID): new macro. (ONIGENC_CONSTRUCT_MBCLEN_NEEDMORE): new macro. (ONIGENC_MBCLEN_CHARFOUND): new macro. (ONIGENC_MBCLEN_INVALID): new macro. (ONIGENC_MBCLEN_NEEDMORE): new macro. (ONIGENC_MBC_ENC_LEN): use ONIGENC_PRECISE_MBC_ENC_LEN. * enc/euc_jp.c: validation implemented. * enc/sjis.c: ditto. * enc/utf8.c: ditto. * string.c (rb_str_inspect): use rb_enc_precise_mbclen for invalid encoding. (rb_str_valid_encoding_p): new method String#valid_encoding?. * io.c (rb_io_getc): use rb_enc_precise_mbclen. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14119 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c: rename primary_encoding -> default_external (encoding).matz2007-12-011-4/+4
| | | | git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14069 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h, encoding.c, re.c, string.c, parse.y: akr2007-11-271-3/+3
| | | | | | | | | | rename ENC_CODERANGE_SINGLE to ENC_CODERANGE_7BIT. rename ENC_CODERANGE_MULTI to ENC_CODERANGE_8BIT. Because single byte 8bit character, such as Shift_JIS 1byte katakana, is represented by ENC_CODERANGE_MULTI even if it is not multi byte. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14027 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_enc_str_asciionly_p): declared.akr2007-11-251-0/+2
| | | | | | | | | | | | | | (rb_enc_str_asciicompat_p): defined. * re.c (rb_reg_initialize_str): use rb_enc_str_asciionly_p. (rb_reg_quote): return ascii-8bit string if the argument is ascii-only to generate encoding generic regexp if possible. (rb_reg_s_union): fix encoding handling. [ruby-dev:32094] * string.c (rb_enc_str_asciionly_p): defined. git-svn-id: http://svn.ruby-lang.org/repos/ruby/trunk@14013 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit (git 2.34.1) at 2026-01-01 01:01:00 +0000