NAME

    Image::DecodeQR::WeChat - Decode QR code(s) from images using the
    OpenCV/WeChat library via XS

VERSION

    Version 2.2

SYNOPSIS

    This module detects and decodes QR code(s) in an input image.

    It provides a Perl interface to the C++ OpenCV/WeChat QR code decoder
    via XS code. OpenCV/WeChat library uses CNN (Convolutional Neural
    Networks) to do this with pre-trained models. It works quite well.

    This module has been tested with OpenCV v4.5.5, v4.8.1, v4.9 and Perl
    v5.32, v5.38 on Linux. But check the CPANtesters matrix on the left for
    all the tests done on this module (although tests may be sparse and
    rare because of the OpenCV dependency).

    The OpenCV/WeChat library is relatively successful even for
    non-orthogonally rotated codes. It remains to be tested fully on the
    minimum size of the code images. 60px was the minimum size with my
    tests. See section "TESTING THE QR CODE DETECTION ALGORITHM" for more
    details.

    Here is some code to get you started:

        use Image::DecodeQR::WeChat;
    
        # decode QR image with convenient "named" params
        my $ret = Image::DecodeQR::WeChat::detect_and_decode_qr({
            # these are required
            'input' => 'input.jpg',
            # optional with defaults:
            # specify a different models dir
            #'modelsdir' => '...',
            # dump results to file(s) whose name
            # is prepended by param 'outbase'
            #'outbase' => 'outs',
            # set this to 1 to also dump images of
            # QR codes detected as well text files with
            # payload and bounding box coordinates
            #'dumpqrimagestofile' => 0,
            # use OpenCV's GUI image viewer to display
            # this needs a non-headless OS and OpenCV highgui
            #'graphicaldisplayresult'' => 0,
            #'verbosity' => 0,
        });
        die "failed" unless $ret;
    
        # we got back an array-of-2-arrays
        # * one contains the QR-code-text (called payload)
        # * one contains bounding boxes, one for each payload
        # we have as many payloads and bounding boxes as
        # are the QR-codes detected (some may have been skipped)
    
        # the number of QR code images found in the input
        # NOTE: the returned array will contain
        # just 2 empty arrays if no QR code was detected
        # so this is the right way:
        my $num_qr_codes_detected = scalar @$payloads;
        
        my ($payloads, $boundingboxes) = @$ret;
        for (0..$#$payloads){
          print "Payload got: '".$payloads->[$_]
            ."' bbox: @{$boundingboxes->[$_]}"
            .".\n";
        }
    
        # Alternatively, a less convenient method to
        # decode a QR code is via the XS sub.
        # It requires that all parameters be specified
        # unlike detect_and_decode_qr() which uses
        # "named" parameters  with defaults.
        my $ret = Image::DecodeQR::WeChat::detect_and_decode_qr_xs(
            # the input image containing one or more QR-codes
            'an-input-image.png',
    
            # the dir with model parameters required by the library.
            # Model files come with this Perl module and are curtesy of WeChat
            # which is part of OpenCV contrib packages.
            # They are installed with this module and their default location
            # is given by Image::DecodeQR::WeChat::modelsdir()
            # Alternatively, you specify here your own model files:
            Image::DecodeQR::WeChat::modelsdir(),
    
            # outbase for all output files, optional = set to undef,
            # if more than one QR-codes were detected then an index will
            # be appended to the filename. And there will be JPEG image files
            # containing the portion of the image which was detected
            # and there will be txt files with QR-code text (payload)
            # and its bounding box. And there will be an overall
            # text file with all payloads. This last one will be
            # printed to STDOUT if no outbase was specified:
            'output.detected',
    
            # verbosity level. 0:mute, 1:C code messages, 10:C+XS code
            10,
    
            # display results in a window with QR codes found highlighted
            # make sure you have an interactive shell and GUI
            1,
    
            # dump image and metadata to files for each QR code detected
            # only if outbase was specified
            1,
        );
        die "failed" unless $ret;
    
        # again, the same data structure returned:
        my ($payloads, $boundingboxes) = @$ret;
        my $num_qr_codes_detected = scalar @$payloads;
        for (0..$#$payloads){
          print "Payload got: '".$payloads->[$_]
            ."' bbox: @{$boundingboxes->[$_]}"
            .".\n";
        }
    
        # Where is it looking for default
        # pre-trained models location ?
        # (they are installed with this module)
        print "my models are in here: ".Image::DecodeQR::WeChat::modelsdir()."\n"
    
        # returns 1 or 0 when OpenCV was compiled with highgui or not
        # and supports GUI display like imshow() which displays an image in a window
        my $has_highgui_support = opencv_has_highgui_xs();

    This code interfaces functions and methods from OpenCV/WeChat library
    (written in C++) for decoding one or more QR code images found embedded
    in images. It's just that: a very thin wrapper of a C++ library written
    in XS. It only interfaces the OpenCV/WeChat library for QR code
    decoding and accommodates its returned data into a Perl array (ref).

    It can detect multiple QR codes embeded in a single image. It has been
    successfully tested with images as small as 60 x 60 pixels.

    The payload (i.e. the QR-code's text) and the coordinates of the
    bounding box around each QR code image detected are returned back as
    Perl array of tuples (i.e. [[QR code text, bounding box]...).

    Optionally, it can output the portion of the input image corresponding
    to each QR-code (that is a sub-image, part of the input image), its
    bounding box and the payload in separate files, useful for debugging
    and identification when multiple QR codes exist in a single input
    image.

FUTURE WORK

    Following the XS code in this module as a guide, it will be trivial to
    interface other parts of the OpenCV library:

        Ιδού πεδίον δόξης λαμπρόν
           (behold a glorious field of glory)

EXPORT

      * detect_and_decode_qr()

      * detect_and_decode_qr_xs()

      * modelsdir()

      * opencv_has_highgui_xs()

SUBROUTINES/METHODS

 detect_and_decode_qr(\%params)

    It tries to detect all the QR codes in the input image (specified by
    its filepath) with one or more QR codes embedded in it.

    It wraps the XS function detect_and_decode_qr_xs(\@params) and replaces
    missing optional parameters with defaults.

    These are the \%params it accepts:

      * input : the filepath of the input image possibly embedded with QR
      code(s). The format of the input image can be anything OpenCV
      supports, see OpenCV's imread()
      <https://docs.opencv.org/4.9.0/d4/da8/group__imgcodecs.html>.

      * modelsdir : optional parameter to specify an alternative models
      location, other than the one installed during the module's
      installation. The current models dir is found using modelsdir(). See
      "MODELS" for more information on what this directory should contain.
      If this parameter is omitted the default value is determined by what
      modelsdir() returns.

      * dumpqrimagestofile : optional flag (0 or 1) to specify whether to
      write results into output files (as well as returning them back as an
      ARRAYref). If this parameter is omitted the default value is zero,
      meaning NO don't dump files.

      These are what the set of output files comprises of:

	* An image of the detected QR code. This is part of the input image
	detailing only the detected QR code. It is saved as a JPEG image
	file.

	* XML output with 1) the detected QR code payload (the text it
	contains), followed by 2) the coordinates of the bounding box of
	the detected QR code. These coordinates are for the input image.
	Here is an example:

          <qrcodes>
            <qrcode>
              <payload>Just another Perl hacker</payload>
              <boundingbox>[9, 1], [409, 1], [409, 401], [9, 401]</boundingbox>
            </qrcode>
          </qrcodes>

      There will be as many pairs of files as the detected QR codes in the
      input image. The filenames will be formed with the outbase parameter
      (which is mandadory if this parameter is set to 1) followed by a
      zero-based index number denoting the sequence of the detected QR code
      (even if there is only one detected QR code, the output files will
      still contain an index number, in this case it will be 0), followed
      by the extentsion .jpg for the image file and .xml for the text file.
      For example $outbase.1.xml and $outbase.1.jpg, where $outbase is the
      specified in the input parameters (see below).

      Additionally, an overall output file whose name is the same as above
      except that it has no index number, will be created to contain the
      payloads of ALL QR codes detected. Each on its own line.

      * outbase : optional parameter which must be specified if
      dumpqrimagestofile is set to 1. If left undef then no results are
      written to files. If specified but dumpqrimagestofile is set to 0,
      then it saves all results into a single file $outbase.xml. If
      specified and dumpqrimagestofile is set to 1 then in addition to the
      overall XML file, for each QR code detected there will be an output
      image as a copy of the input with the detected QR code outlined and
      an output XML file with metadata (the payload and bounding box
      coordinates).

      WARNING: if you set dumpqrimagestofile to 1 then as many output
      images as the detected QR codes will be created and all will be a
      copy of the input image, that can be a lot of large images ...

      outbase will be prepended to the name of each of the output result
      files. outbase may contain directory components but make sure they do
      exist because they will not be created.

      * graphicaldisplayresult : optional flag which applies only in the
      case where the underlying OpenCV installation contains the highgui
      GUI component which allows for displaying images. If that component
      exists (which can be determined by calling opencv_has_highgui_xs())
      then setting this parameter to 1 will also display image results (the
      detected QR codes) into a GUI window as the detection process is
      happening. The user must close this window in order for detection
      process to continue until the script exits. The default value is
      zero, meaning NO do not display any output images.

      * verbosity : set this to a positive integer in order for the program
      to print debugging messages to standard output. Verbosity increases
      as this value increases. With zero, the default value, being
      completely mute.

  Returned data by detect_and_decode_qr()

    On success, it returns results back as an ARRAYref of 2-item-arrays
    (tuples) containing:

      * The text of the decoded QR code detected.

      * The coordinates of the bounding box around the QR code image
      detetced.

    It returns an array of two zero-length arrays if no QR codes were
    detected.

    It returns undef on failure.

    Noting all the above, here is a way of calling it, checking its success
    and iterating over all the QR codes data returned:

      my $ret = detect_and_decode_qr(\%params);
      die "failed to detect_and_decode_qr()" unless defined $ret;
      my ($payloads, $bounding_boxes) = @$ret;
      # the number of detected QR codes (can be zero!):
      my $num_qr_codes_detected = scalar @$payloads;
      for ( 0 .. ($num_qr_codes_detected-1) ){
        print "payload: ".$payloads->[$_]."\n";
        print "bbox: ".$bounding_boxes->[$_]."\n";
      }

 detect_and_decode_qr_xs(infile, modelsdir, outbase, verbosity,
 graphicaldisplayresult, dumpqrimagestofile)

    It tries to detect all the QR codes in the input image (specified by
    its filepath) with one or more QR codes embedded in it.

    This is an XS function (which can be called safely by a Perl script)
    wrapped by detect_and_decode_qr(\%params) which is more convenient as
    it replaces missing parameters with defaults, unlike this function
    which expects all the parameters to be specified.

    These are the @params it accepts, in this order:

      * input : the filepath to the input image possibly containing QR
      codes. The format of the input image can be anything OpenCV supports,
      see OpenCV's imread()
      <https://docs.opencv.org/4.5.5/d4/da8/group__imgcodecs.html>.

      * modelsdir : the path to the directory conntaining the pre-trained
      CNN models which are used for the QR code detection. This parameter
      must be specified. The default models dir is given by calling
      modelsdir(). You can pass the return of modelsdir() as this parameter
      if you do not have your own models.

      * outbase : the string prepended to all the output files. It must be
      specified (i.e. not left undef) only when dumpqrimagestofile is set
      to 1.

      * verbosity : set it to a positive integer to get debugging output to
      standard output. Set it to zero for mute operation.

      * graphicaldisplayresult : set it to 1 to display image results into
      an image viewer provided by OpenCV (highgui and imgview()). Set it to
      0 for not displaying anything. If set to 1, OpenCV must contain the
      highgui library component which is optional. OpenCV installations to
      headless servers most likely will not contain this component.
      Although it is possible to install it.

      * dumpqrimagestofile : set it to 1 in order to dump results (text and
      images) to output files whose names are prepended by outbase. Set it
      to 0 for not saving any results to files. Results will still be
      returned back by this function as an array.

    It returns exactly the same results as detect_and_decode_qr(), see
    "Returned data by detect_and_decode_qr()" for details.

 modelsdir()

    It returns the path to the default location of the directory containing
    the pre-trained CNN models.

 opencv_has_highgui_xs()

    It returns 0 or 1 depending whether OpenCV's highgui library component
    was detected during this module's installation. If the result is 1 then
    the component is installed in your system and the
    graphicaldisplayresult parameter to both detect_and_decode_qr(\%params)
    and detect_and_decode_qr_xs(@params) can be set to 1. See CAVEATS for
    the efficacy of this subroutine.

COMMAND LINE SCRIPT

      image-decodeqr-wechat.pl --input image-with-qr-code.jpg
    
      image-decodeqr-wechat.pl --help

    A CLI script is provided and will be installed by this module. Basic
    usage is as above. Here is its usage:

      Usage : script/image-decodeqr-wechat.pl <options>
    
      where options are:
    
        --input F :
          the filename of the input image
          which supposedly contains QR codes to be detected.
    
        --modelsdir M :
          optionally use your own models contained
          in this directory instead of the ones
          this program was shipped with.
    
        --outbase O :
          basename for all output files
          (if any, depending on whether --dumpqrimagestofile is on).
    
        --verbosity L :
          verbosity level, 0:mute, 1:C code, 10:C+XS code.
    
        --graphicaldisplayresult :
          display a graphical window with input image
          and QR codes outlined. Using --dumpqrimagestofile
          and specifying --outbase, images and payloads and
          bounding boxes will be saved to files, if you do
          not have graphical interface.
    
        --dumpqrimagestofile :
          it has effect only of --outbase was specified. Payloads,
          Bounding Boxes and images of each QR-code detected will
          be saved in separate files.

PREREQUISITES

      * OpenCV library with contributed modules is required. The
      contributed modules must include the QR-code decoder library by
      WeChat. OpenCV must also contain the include dir (headers) - just
      saying.

      * A C++ compiler must be installed in your system.

      * Optionally, pkg-config or cmake must be installed in order to aid
      discovering the location of OpenCV's library and include dir. If you
      don't have these installed then you must manually set environment
      variables OPENCV_LDFLAGS and OPENCV_CFLAGS to point to those paths
      and then attempt to install this module (e.g. perl Makefile.PL; make
      all ; make install)

      * Optionally, OpenCV can contain the Highgui library so that output
      images can be displayed in their own window. But this is superfluous,
      because the basic operation of this module allows for saving output
      files to disk.

INSTALLING OpenCV

    In my case installing OpenCV using Linux's package manager (dnf,
    fedora) was not successful with default repositories. It required to
    add another repository (rpmfusion) which wanted to install its own
    versions of packages I already had. So I prefered to install OpenCV
    from sources. This is the procedure I followed:

      * Download OpenCV sources and also its contributed modules.

      * Extract the sources and change to the source dir.

      * From within the source dir extract the contrib archive.

      * Create a build dir and change to it.

      * There are two ways to make cmake just tolerable: cmake-gui and
      ccmake. The former is a full-gui interface to setting the billion
      cmake variables. Use it if you are on a machine which offers a GUI
      like this: cmake-gui .. If you are on a headless or remote host
      possibly over telnet or ssh then do not despair because ccmake is the
      CLI, curses-based equivalent to cmake-gui, use it like: ccmake ../
      (from within the build dir).

      * Once on either of the cmake GUIs, first do a configure, then check
      the list of all variables (you can search on both, for searching in
      the CLI, press / and then n for next hit) to suit you and then
      generate, quit and VERBOSE=1 make -j4 all

      * I guess, cmake variables you want to modify are
      OPENCV_EXTRA_MODULES_PATH and turn ON OPENCV_ENABLE_NONFREE and
      anything that has to do with CNN or DNN. If you have CUDA installed
      and a CUDA-capable GPU then enable CUDA (search for CUDA string to
      find the variable(s)). Also, VTK, Ceres Solver, Eigen3, Intel's TBB,
      CNN, DNN etc. You need to install all these additional packages
      first, before finally compiling OpenCV.

      * I had a problem with compiling OpenCV with a GUI (the highgui) on a
      headless host. So, I just disabled it. That's easy to achieve during
      the above.

      * I have successfully installed this module on a system with
      CUDA-capable GPU (with CUDA 10.2 installed) host and, also, on a
      headless remote host with no GPU or basic. In general, CUDA is not
      required for building this module. It is just an addition for making
      things run faster, possibly. OpenCV is responsible for detecting and
      utilising GPU processing via CUDA.

      * It is also possible to download a binary distribution of OpenCV
      with developer files (e.g. header files and perhaps a pkg-config or
      cmake configuration file). Just make sure that it supports all the
      things I mentioned above.

      * If all else fails, then add rpmfusion repository to your Linux's
      package manager and then add package OpenCV, developer version. Make
      sure there is the WeChat OpenCV library too. If there is not, perl
      Makefile.PL will complain and fail.

    Your mileage may vary.

    If you are seriously in need of installing this module then consider
    migrating to a serious operating system such as Linux as your first
    action.

INSTALLING THIS MODULE

    This module depends on the existence of the OpenCV library with all the
    extensions and contributed modules mentioned in section
    "PREREQUISITES".

    Detecting where this library is located in your system is the weakest
    link in the installation process of this module. Makefile.PL contains
    code to do this with pkg-config or cmake. If these fail, it will look
    for ENVironment variables: OPENCV_LDFLAGS and OPENCV_CFLAGS, which
    should contain the CFLAGS (for example: -I/usr/include/opencv4/) and
    LDFLAGS (for example: -L/usr/lib64 -lopencv_world). Set these variables
    manually prior installation if the automatic methods mentioned above
    fail.

    One last thing to check is that if your OpenCV installation (developer
    version) was correct, there should be a pkg-config file, perhaps in
    /usr/lib64/pkgconfig/opencv4.pc or
    /usr/local/lib64/pkgconfig/opencv4.pc. This file details all the CFLAGS
    and LDFLAGS and should be found by Makefile.PL if it is in a standard
    location, or adjust the list of paths in environment variable
    PKG_CONFIG_PATH which is where pkg-config searches for these files.

TESTING THE QR CODE DETECTION ALGORITHM

    You will want to produce QR codes in order to assess how well the
    algorithm (basically the CNN pre-trained models supplied by
    OpenCV/WeChat) detects codes and find out the minimum size, quality,
    rotation angles etc. for optimal detection.

    In producing test images from an original QR code, with software like
    the GIMP <https://www.gimp.org/>, one should be aware of the
    distortions caused by transforms such as scale and rotation to the
    final QR code images, rotation in particular. Add certain enhancements
    to the final image to "look good" and the resultant image looks like QR
    code but it is not. Failure of the library on such artificially
    produced images would be somehow expected.

    Instead I would suggest testing with images which have been scanned
    with a QR code image attached to them in random angles and zoom
    factors. Using a photocopier or a scanner.

    My use case was to process scanned images with a glued-in QR code tag
    which is detected and archive the document from the scanner to the
    appropriate files.

UNIT TESTING

    There are two sets of tests which can be performed before installation
    of this module. The first test is done by default and can be run with:

      perl Makefile.PL
      make all
      make test

    The second set of tests is what is called author tests and is
    optionally run with:

      perl Makefile.PL
      make all
      prove -bl xt

    In both sets setting the environment variable TEMP_DIRS_KEEP to 1 will
    keep all temporary files created so that inspection of output files is
    possible. By default all temporary files created during testing are
    erased on test's exit.

MODELS

    The OpenCV/WeChat QR code detector algorithm uses CNN (Convolutional
    Neural Networks) which are required to be trained first with data
    containing example QR code images. The detector kindly provided by
    OpenCV/WeChat already contains trained models (see "LICENSE AND
    COPYRIGHT" for license) which are also contained in and distributed
    with this module. These pre-trained models are installed as part of
    this module, along with everything else. You do not need to download
    them manually.

    The pre-trained models can be found here
    <https://github.com/WeChatCV/opencv_3rdparty> where it also shows their
    MD5 signatures. Their total size is about 1 MB.

    The models directory must contain four files: detect.caffemodel,
    detect.prototxt, sr.caffemodel and sr.prototxt.

    Use modelsdir() to get the location of the installed models.

IMPLEMENTATION DETAILS

    This code demonstrates how to call OpenCV (modern OpenCV v4) C++
    methods using the technique suggested by Botje @ #perl in order to
    avoid all the function, macro, data structures name clashes between
    Perl and OpenCV (for example seed(), do_open(), do_close() and most
    notably struct cv and namespace cv in Perl and OpenCV respectively).

    The trick suggested is to put all the OpenCV-calling code in a separate
    C++ file and provide high-level functions to be called by XS. So that
    the XS code does not see any OpenCV header files.

    Makefile.PL will happily compile any .c and/or .cpp files found in the
    dir it resides by placing OBJECT => '$(O_FILES)' in %WriteMakefileArgs.
    And will have no problems with specifying also these:

        CC      => 'g++',
        LD      => 'g++',
        XSOPT   => '-C++',

    With one caveat, g++ compiler will mangle the names of the functions
    when placing them in the object files. And that will cause XSLoader to
    report missing and undefined symbols.

    The cure to this is to wrap any function you want to remain unmangled
    between these two blocks:

        #ifdef __cplusplus
        extern "C" {
        #endif

    and

        #ifdef __cplusplus
        } //extern "C" {
        #endif

    This only need happen in the header file:  wechat_qr_decode_lib.hpp 
    and in the XS file where the Perl headers are included.

CAVEATS

    Checking for whether local OpenCV installation has highgui support is
    currently very lame. It tries to detect it with three methods (see
    find_if_opencv_highgui_is_supported() in Makefile.PL for the
    implementation)

      * Use FFI::CheckLib::find_lib to check if library opencv_highgui
      exists.

      * Search Include dirs for file opencv2/highgui.hpp. This is the most
      OS-agnostic method.

      * Use Devel::CheckLib::check_lib to check if a sample C program can
      link to the opencv_highgui library. This requires a C compiler.

    Note that DynaLoader (or FFI::CheckLib which uses it) can search for
    symbols in any library (e.g. highgui library should contain function
    imshow() in libopencv_highgui or libopencv_world). This would have been
    the most straight-forward way but alas, these are C++ libraries and the
    contained function names are mangled to weird function names like:

        ZN2cv3viz6imshowERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_11_InputArrayERKNS_5Size_IiEE()

    There's an imshow string in there but without a regex symbol-name
    search the symbol can not be detected. Currently, DynaLoader (which is
    called by FFI::CheckLib) does not provide a regular expression symbol
    name matching, only exact.

AUTHOR

    Andreas Hadjiprocopis, <bliako at cpan.org>

BUGS

    Please report any bugs or feature requests to bug-image-decodeqr-wechat
    at rt.cpan.org, or through the web interface at
    https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Image-DecodeQR-WeChat.
    I will be notified, and then you'll automatically be notified of
    progress on your bug as I make changes.

SUPPORT

    You can find documentation for this module with the perldoc command.

        perldoc Image::DecodeQR::WeChat

    You can also look for information at:

      * RT: CPAN's request tracker (report bugs here)

      https://rt.cpan.org/NoAuth/Bugs.html?Dist=Image-DecodeQR-WeChat

      * Review this module at PerlMonks

      https://www.perlmonks.org/?node_id=21144

      * Search CPAN

      https://metacpan.org/release/Image-DecodeQR-WeChat

ACKNOWLEDGEMENTS

      * The great Open Source OpenCV <https://opencv.org/> image processing
      library and its contributed module WeChat QRDetector
      <https://docs.opencv.org/4.x/dd/d63/group__wechat__qrcode.html> which
      form the backbone of this module and do all the heavy lifting.

      * Botje and xenu at #perl for help.

      * Jiro Nishiguchi (JIRO ) whose (obsolete with modern - at the time
      of writing - OpenCV) module Image::DecodeQR serves as the skeleton
      for this module.

      * Thank you! to all those who responded to this SO question
      https://stackoverflow.com/questions/71402095/perl-xs-create-and-return-array-of-strings-char-taken-from-calling-a-c-funct

      * The Hackers of Free Software.

LICENSE AND COPYRIGHT

    This software is Copyright (c) 2022 by Andreas Hadjiprocopis.

    This is free software, licensed under:

      The Artistic License 2.0 (GPL Compatible)

    The OpenCV/WeChat CNN trained models redistributed with this module are
    licensed under

      The Apache License Version 2.0

    See the full licence here
    <https://github.com/WeChatCV/opencv_3rdparty/commit/3487ef7cde71d93c6a01bb0b84aa0f22c6128f6b>.

HUGS

    !Almaz!

