=encoding UTF-8

=head1 NAME

Data::Kanji::Kanjidic - parse the "kanjidic" kanji data file

=head1 SYNOPSIS

    use Data::Kanji::Kanjidic 'parse_kanjidic';
    my $kanji = parse_kanjidic ('/path/to/kanjidic');
    for my $k (keys %$kanji) {
        print "$k has radical number $kanji->{$k}{radical}.\n";
    }


=head1 DESCRIPTION

This module's basic function is to read the entire Kanjidic file into
memory and create a data structure from it. The user then uses this
data structure for whatever purpose. Parsing Kanjidic takes a second
or two. Here the Kanjidic file is the as-downloaded text file in the
old format, rather than the new-format XML file.

There are several examples of the module's use in the F<examples>
directory of the distribution, including all the code in the module's
documentation.

=head1 FUNCTIONS

=head2 parse_kanjidic

    
    use Data::Kanji::Kanjidic 'parse_kanjidic';
    my $kanjidic = parse_kanjidic ('/path/to/kanjidic');
    print "@{$kanjidic->{猫}{english}}\n";
    # This prints out "cat".
    


The input is the file name where Kanjidic may be found. The return
value is a hash reference. The keys of this hash reference are kanji,
encoded as Unicode. Each of the values of the hash reference are
entries corresponding to the kanji in the keys. Each value represents
one line of Kanjidic. Each is a hash reference, with the keys
described in L</parse_entry>.

This function assumes that the kanjidic file is encoded using the
EUC-JP encoding.

=head2 parse_entry

    my %values = parse_entry ($line);

Parse one line of Kanjidic. The input is one line from Kanjidic,
encoded as Unicode. The return value is a hash containing each field
from the line.

The possible keys and values of the returned hash are as
follows. Values are scalars unless otherwise mentioned.

=over

=item kanji

The kanji itself (the same as the key).

=item jiscode

The JIS code for the kanji in hexadecimal. This is a two-byte number
which identifies the kanji in the JIS X 0208 encoding scheme. See
L<http://www.sljfaq.org/afaq/encodings.html#encodings-JIS-X-0208>. The
JIS value is the second value in Kanjidic after the kanji in encoded
form and before the Unicode code point.



=item B

Bushu (radical as defined by the Nelson kanji dictionary).

=item C

Classic radical (the usual radical, where this is different from the Nelson radical).

=item DB

Japanese for Busy People textbook numbers.

=item DC

The index numbers used in "The Kanji Way to Japanese Language Power" by Dale Crowley.

=item DF

"Japanese Kanji Flashcards", by Max Hodges and Tomoko Okazaki.

=item DG

The index numbers used in the "Kodansha Compact Kanji Guide".

=item DH

The index numbers used in the 3rd edition of "A Guide To Reading and Writing Japanese" edited by Kenneth Hensall et al.

=item DJ

The index numbers used in the "Kanji in Context" by Nishiguchi and Kono.

=item DK

The index numbers used by Jack Halpern in his Kanji Learners Dictionary.

=item DM

The index numbers from the French-language version of "Remembering the kanji".

=item DO

The index numbers used in P.G. O'Neill's Essential Kanji.

=item DR

The codes developed by Father Joseph De Roo, and published in his book "2001 Kanji" (Bonjinsha).

=item DS

The index numbers used in the early editions of "A Guide To Reading and Writing Japanese" edited by Florence Sakade.

=item DT

The index numbers used in the Tuttle Kanji Cards, compiled by Alexander Kask.

=item E

The numbers used in Kenneth Henshall's kanji book.

=item F

Frequency of kanji.

The following example program prints a list of kanji from most to least
frequently used.

    
    use Data::Kanji::Kanjidic 'parse_kanjidic';
    my $kanji = parse_kanjidic ('/path/to/kanjidic');
    my @sorted;
    for my $k (keys %$kanji) {
        if ($kanji->{$k}->{F}) {
            push @sorted, $kanji->{$k};
        }
    }
    @sorted = sort {$a->{F} <=> $b->{F}} @sorted;
    for (@sorted) {
        print "$_->{kanji}: $_->{F}\n";
    }
    


=item G

Year of elementary school this kanji is taught.

This field is also used by kanjidic to give information on whether the
kanji is part of the Joyo or Jinmeiyo Kanji sets. If the grade is
between 1 and 8, the kanji is part of the Joyo Kanji. (See
L<http://www.sljfaq.org/afaq/jouyou-kanji.html> for details of the
Joyo Kanji.) If the grade is 9 or 10, then the kanji is not part of
the Joyo kanji, but it is part of the Jinmeiyo Kanji. (See
L<http://www.sljfaq.org/afaq/jinmeiyou-kanji.html>.)

See also L</grade>.


=item H

Number in Jack Halpern dictionary.

=item I

The Spahn-Hadamitzky book number.

=item IN

The Spahn-Hadamitzky kanji-kana book number.

=item J

Japanese proficiency test level.

=item K

The index in the Gakken Kanji Dictionary (A New Dictionary of Kanji Usage).

=item L

Code from "Remembering the Kanji" by James Heisig.

=item MN

Morohashi index number.

=item MP

Morohashi volume/page.

=item N

Nelson code from original Nelson dictionary.

=item O

The numbers used in P.G. O'Neill's "Japanese Names".
This may take multiple values, so the value is an array reference.

=item P

SKIP code.

=item Q

Four-corner code.
This may take multiple values, so the value is an array reference.

=item S

Stroke count.
This may take multiple values, so the value is an array reference.

=item T

SPECIAL.

=item U

Unicode code point as a hexadecimal number.

=item V

Nelson code from the "New Nelson" dictionary.
This may take multiple values, so the value is an array reference.

=item W

Korean pronunciation.
This may take multiple values, so the value is an array reference. 

The following example program prints a list of Korean pronunciations,
romanized (requires L<Lingua::KO::Munja>).

    use Data::Kanji::Kanjidic 'parse_kanjidic';
    use Lingua::KO::Munja ':all';
    my $kanji = parse_kanjidic ($ARGV[0]);
    for my $k (sort keys %$kanji) {
        my $w = $kanji->{$k}->{W};
        if ($w) {
            my @h = map {'"' . hangul2roman ($_) . '"'} @$w;
            print "$k is Korean ", join (", ", @h), "\n";
        }
    }


=item X

Cross reference.

=item XDR

De Roo cross-reference.
This may take multiple values, so the value is an array reference.

=item XH

Cross-reference.
This may take multiple values, so the value is an array reference.

=item XI

Cross-reference.

=item XJ

Cross-reference.
This may take multiple values, so the value is an array reference.

=item XN

Nelson cross-reference.
This may take multiple values, so the value is an array reference.

=item XO

Cross-reference.

=item Y

Pinyin pronunciation.
This may take multiple values, so the value is an array reference.

=item ZBP

MISCLASSIFICATIONrp.
This may take multiple values, so the value is an array reference.

=item ZPP

MISCLASSIFICATIONpp.
This may take multiple values, so the value is an array reference.

=item ZRP

MISCLASSIFICATIONrp.
This may take multiple values, so the value is an array reference.

=item ZSP

MISCLASSIFICATIONsp.
This may take multiple values, so the value is an array reference.

=item radical

This is the Kangxi radical of the kanji. This overrides Kanjidic's
preference for the Nelson radical. In other words, this is the same as
the L</B> field for most kanji, but if a L</C> field exists, this is the value
of the C field rather than the B field.

=item kokuji

This has a true value (1) if the character is marked as a "kokuji" in
Kanjidic. See L<http://www.sljfaq.org/afaq/kokuji.html>.

=item english

This contains an array reference to the English-language meanings
given in Kanjidic. It may be undefined, if there are no
English-language meanings listed.

    # The following "joke" program converts English into kanji.
    
    # Call it with two arguments, first the location of kanjidic, and
    # second a file of English text to "corrupt":
    #
    # ./english-to-kanji.pl /where/is/kanjidic english-text-file
    
    use Data::Kanji::Kanjidic 'parse_kanjidic';
    use Convert::Moji 'make_regex';
    my $kanji = parse_kanjidic ($ARGV[0]);
    my %english;
    for my $k (keys %$kanji) {
        my $english = $kanji->{$k}->{english};
        if ($english) {
            for (@$english) {
                push @{$english{$_}}, $k;
            }
        }
    }
    my $re = make_regex (keys %english);
    open my $in, "<", $ARGV[1] or die $!;
    while (<$in>) {
        s/\b($re)\b/$english{$1}[int rand (@{$english{$1}})]/ge;
        print;
    }
    


Given input like this,

    This is an example of the use of "english-to-kanji.pl", a program which
    converts English words into kanji. This may or may not be regarded as a
    good idea. What do you think?


it outputs this:

    This is an 鑒 之 彼 使 之 "english負to負kanji.pl", a program 孰
    converts 英 辭 into kanji. This 得 将 得 無 跨 regarded as a
    臧 見. What 致 尓 憶?


=item onyomi

This is an array reference which contains the I<on'yomi> (音読) of the
kanji. (See L<http://www.sljfaq.org/afaq/kanji-pronunciation.html>.)
It may be undefined, if no I<on'yomi> readings are listed. The
I<on'yomi> readings are in katakana, as per Kanjidic itself. It is
encoded in Perl's internal Unicode encoding.

The following example prints a list of kanji which have the same on'yomi:

    use Data::Kanji::Kanjidic 'parse_kanjidic';
    my $kanji = parse_kanjidic ($ARGV[0]);
    my %all_onyomi;
    for my $k (keys %$kanji) {
        my $onyomi = $kanji->{$k}->{onyomi};
        if ($onyomi) {
            for my $o (@$onyomi) {
                push @{$all_onyomi{$o}}, $k;
            }
        }
    }
    for my $o (sort keys %all_onyomi) {
        if (@{$all_onyomi{$o}} > 1) {
            print "Same onyomi 「$o」 for 「@{$all_onyomi{$o}}」!\n";
        }
    }


=item kunyomi

This is an array reference which contains the I<kun'yomi> (訓読) of
the kanji. (See
L<http://www.sljfaq.org/afaq/kanji-pronunciation.html>.) It may be
undefined, if no I<kun'yomi> readings are listed. The I<kun'yomi>
readings are in hiragana, as per Kanjidic itself. It is encoded in
Perl's internal Unicode encoding.

=item nanori

This is an array reference which contains I<nanori> (名乗り) readings
of the character. It may be undefined, if no I<nanori> readings are
listed. The I<nanori> readings are in hiragana, as per Kanjidic
itself. They are encoded in Perl's internal Unicode encoding.

=item morohashi

This is a hash reference containing data on the kanji's location in
the Morohashi 'Dai Kan-Wa Jiten' kanji dictionary. The hash reference
has the following keys.

=over

=item volume

The volume number of the character.

=item page

The page number of the character.

=item index

The index number of the character.

=back

If there is no information, this remains unset.

For example, to print all the existing values,

    
    use Data::Kanji::Kanjidic 'parse_kanjidic';
    use FindBin;
    my $kanji = parse_kanjidic ("/path/to/kanjidic");
    for my $k (sort keys %$kanji) {
        my $mo = $kanji->{$k}->{morohashi};
        if ($mo) {
            print "$k: volume $mo->{volume}, page $mo->{page}, index $mo->{index}.\n";
        }
    }
    


=back

For detailed explanations of these codes, see L</Kanjidic>.

=head2 kanjidic_order

    my @order = kanjidic_order ($kanjidic_ref);

Given an input of the return value of L</parse_kanjidic>, this returns
a list of the kanji sorted by the stroke order (the S field) of the
elements of C<\%kanjidic>. Elements with the same stroke order are
sorted by radical (the B field). This also adds a new field "kanji_id"
to each of them so that the order can be reconstructed when referring
to elements.

This replicates the ordering of kanji in a kanji dictionary's index by
stroke order. See
L<http://www.sljfaq.org/afaq/using-a-kanji-dictionary.html> for more
on kanji dictionary ordering.

=head2 grade

    my $grade2 = grade ($kanjidic_ref, 2);

Given a school grade such as C<2> above, and the return value of
L</parse_kanjidic>, C<$kanjidic_ref>, return an array reference
containing a list of all of the kanji from that grade. (See
L<http://www.sljfaq.org/afaq/education.html> for details of the
Japanese education system.)

The following example prints a list of the kanji from each school
grade to standard output:

    
    use Data::Kanji::Kanjidic qw/parse_kanjidic grade/;
    my $kanjidic = parse_kanjidic ('/path/to/kanjidic');
    for my $grade (1..6) {
        my $list = grade ($kanjidic, $grade);
        print "Grade $grade:\n\n";
        my $count = 0;
        for (sort @$list) {
            print "$_ ";
            $count++;
            if ($count % 20 == 0) {
                print "\n";
            }
        }
        print "\n";
    }
    


=head2 stroke_radical_jis_order

    @list = sort { stroke_radical_jis_order ($kanjidic, $a, $b) } @list;

This is a comparison function which sorts kanji C<$a> and C<$b>
according to stroke count, the L</S> field. If the stroke count is
identical, it sorts them according to L</radical>. If both the stroke
count and radical are the same, it sorts them according to
L</jiscode>.

=head1 SEE ALSO

=head2 Other Perl modules

=over

=item L<Lingua::JP::Kanjidic>

This module parses an old version of F<kanjidic>.

=back

=head2 Kanjidic

The official description of kanjidic is in
L<http://www.csse.monash.edu.au/~jwb/kanjidic.html>.  To download
F<kanjidic>, please go to this web page and then download it from the
link provided.

=head1 AUTHOR

Ben Bullock, <bkb@cpan.org>

=head1 COPYRIGHT & LICENCE

This package and associated files are copyright (C) 2012-2013 Ben
Bullock.

You can use, copy, modify and redistribute this package and associated
files under the Perl Artistic Licence or the GNU General Public
Licence.


