Galen Charlton [Mon, 30 Jul 2012 18:47:37 +0000]
fix error message displayed if exception file cannot be opened
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Mon, 30 Jul 2012 17:48:51 +0000]
quiet a couple warnings
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Mon, 30 Jul 2012 17:42:23 +0000]
utility script to prepare a file of MARCXML records for fingerprinter
Given a two-column tab-delimited text file contain bib IDs and MARCXML, produces
a MARCXML file with the bib IDs in 903 fields.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Mon, 30 Jul 2012 17:32:35 +0000]
improve marc_cleanup's parsing of MARCXML files
In particular, ones produced by use of MARC::File::XML->as_xml_record(),
which inserts namespace attributes in the record element.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Tue, 24 Jul 2012 19:31:08 +0000]
use PL/PerlU rather than PL/Perl
The trusted version of PL/Perl is not enabled by
default if you follow the installation instructions
for recent versions of Evergreen.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Tue, 24 Jul 2012 19:25:58 +0000]
simple stored function to check parseability of MARCXML blobs
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Lebbeous Fogle-Weekley [Fri, 20 Jul 2012 14:34:48 +0000]
Very rough tool to help parse serctl.data from a Symphony export
Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Galen Charlton [Mon, 16 Jul 2012 16:10:48 +0000]
add more copyright and license statements
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Wed, 11 May 2011 15:55:25 +0000]
Add a header line to explain the fields
git-svn-id: svn://nox.esilibrary.com/migration-tools@734
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Mon, 7 Mar 2011 18:15:48 +0000]
Some city/state data has no comma
git-svn-id: svn://nox.esilibrary.com/migration-tools@725
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 13 Oct 2010 15:44:28 +0000]
Now with address line2!
git-svn-id: svn://nox.esilibrary.com/migration-tools@711
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 30 Jun 2010 13:56:10 +0000]
Better parsing of city/state
git-svn-id: svn://nox.esilibrary.com/migration-tools@703
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Fri, 25 Jun 2010 15:37:37 +0000]
Don't clobber user_category1 and user_category2
git-svn-id: svn://nox.esilibrary.com/migration-tools@702
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Thu, 10 Jun 2010 20:00:51 +0000]
Now extracting .COUNTY. fields from addresses
git-svn-id: svn://nox.esilibrary.com/migration-tools@700
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 9 Jun 2010 14:08:45 +0000]
Reinserting a missing space that was causing trouble
git-svn-id: svn://nox.esilibrary.com/migration-tools@698
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 26 May 2010 15:03:48 +0000]
Fixed a bug that improperly parsed 'III' as a suffix.
git-svn-id: svn://nox.esilibrary.com/migration-tools@694
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 26 May 2010 15:00:05 +0000]
Fixed a bug that could prevent name parsing in some cases.
If the name includes an undotted abbreviation (Mrs, Jr) immediately followed by a comma,
the previous version was removing the comma, which got in the way of parsing that name.
git-svn-id: svn://nox.esilibrary.com/migration-tools@693
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Tue, 22 Dec 2009 14:28:43 +0000]
First working draft of analyze_zips.sql
git-svn-id: svn://nox.esilibrary.com/migration-tools@653
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 16 Dec 2009 21:30:45 +0000]
Regex fix was a bug; this undoes it
git-svn-id: svn://nox.esilibrary.com/migration-tools@651
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 16 Dec 2009 21:02:26 +0000]
Fixed regex to handle lines that begin with a dot and something other than a capital letter (i.e., not a field label)
git-svn-id: svn://nox.esilibrary.com/migration-tools@650
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Fri, 6 Nov 2009 16:51:51 +0000]
Minor fix to a comment line
git-svn-id: svn://nox.esilibrary.com/migration-tools@637
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Fri, 6 Nov 2009 16:47:34 +0000]
New script to extract lost items from users.data, when present
git-svn-id: svn://nox.esilibrary.com/migration-tools@636
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Fri, 6 Nov 2009 16:06:23 +0000]
Correctly parsing Unicorn patron names and addresses
git-svn-id: svn://nox.esilibrary.com/migration-tools@635
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Mon, 2 Nov 2009 21:12:03 +0000]
Unicorn charge.data to TSV parser
git-svn-id: svn://nox.esilibrary.com/migration-tools@634
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Mon, 2 Nov 2009 15:17:22 +0000]
Unicorn bill parser finished
git-svn-id: svn://nox.esilibrary.com/migration-tools@633
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Fri, 30 Oct 2009 19:40:47 +0000]
Unfinished draft of Unicorn bill converter
git-svn-id: svn://nox.esilibrary.com/migration-tools@632
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Fri, 30 Oct 2009 18:55:39 +0000]
Utter pedantry? Perhaps, but misleading comments are bugs too.
git-svn-id: svn://nox.esilibrary.com/migration-tools@630
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Tue, 27 Oct 2009 20:18:57 +0000]
First draft mark two (with syntax errors fixed)
git-svn-id: svn://nox.esilibrary.com/migration-tools@628
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Tue, 27 Oct 2009 20:10:40 +0000]
First finished draft of new unicorn_patrons_to_tsv.pl
git-svn-id: svn://nox.esilibrary.com/migration-tools@627
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Tue, 27 Oct 2009 19:57:22 +0000]
Finished the bit that processes the records. Next up: actual output.
git-svn-id: svn://nox.esilibrary.com/migration-tools@626
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Tue, 27 Oct 2009 17:39:05 +0000]
Reforking unicorn importer; this is an unfinished version of a patron importer
git-svn-id: svn://nox.esilibrary.com/migration-tools@624
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Mon, 26 Oct 2009 18:11:29 +0000]
Cleaning up older copy of unicorn_to_tsv.pl
git-svn-id: svn://nox.esilibrary.com/migration-tools@623
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Mon, 26 Oct 2009 18:05:56 +0000]
Unicorn TSV-maker can now handle bill.data and charge.data too.
git-svn-id: svn://nox.esilibrary.com/migration-tools@621
eee7cc8d-164e-4af6-8e1b-
092a69004917
Jason Etheridge [Wed, 9 Sep 2009 22:00:28 +0000]
change \r\n to \n if they occur
git-svn-id: svn://nox.esilibrary.com/migration-tools@617
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Tue, 11 Aug 2009 12:20:22 +0000]
First draft of Unicorn patron-file converter. No known bugs, but memory linear (at least) with number of patrons.
git-svn-id: svn://nox.esilibrary.com/migration-tools@606
eee7cc8d-164e-4af6-8e1b-
092a69004917
Ben Ostrowsky [Wed, 16 Dec 2009 21:32:00 +0000]
And this turns off all the STDERR verbosity. Sheesh; sorry.
git-svn-id: svn://nox.esilibrary.com/migration-tools@652
eee7cc8d-164e-4af6-8e1b-
092a69004917
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Wed, 16 Dec 2009 21:30:45 +0000]
Regex fix was a bug; this undoes it
git-svn-id: svn://nox.esilibrary.com/migration-tools@651
eee7cc8d-164e-4af6-8e1b-
092a69004917
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Wed, 16 Dec 2009 21:02:26 +0000]
Fixed regex to handle lines that begin with a dot and something other than a capital letter (i.e., not a field label)
git-svn-id: svn://nox.esilibrary.com/migration-tools@650
eee7cc8d-164e-4af6-8e1b-
092a69004917
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Fri, 30 Oct 2009 19:18:51 +0000]
FORM=foo should not be ignored
git-svn-id: svn://nox.esilibrary.com/migration-tools@631
eee7cc8d-164e-4af6-8e1b-
092a69004917
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Mon, 26 Oct 2009 18:10:32 +0000]
unicorn_to_tsv.pl has been generalized to use non-user-specific wording
git-svn-id: svn://nox.esilibrary.com/migration-tools@622
eee7cc8d-164e-4af6-8e1b-
092a69004917
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Mon, 16 Jul 2012 15:53:35 +0000]
add copyright statement and GPL2+ license statement
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Mon, 16 Jul 2012 15:43:53 +0000]
make cleanup_merge_map.pl executable
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Galen Charlton [Mon, 16 Jul 2012 15:43:06 +0000]
remove .csvignore
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Wed, 22 Feb 2012 20:30:10 +0000]
Three variations on apply_circ_matrix: _before_20 spells it recuring instead of recurring, _after_20 uses the matchpoint structure, and the unmakred case (for now, at least) does neither.
Ben Ostrowsky [Tue, 24 Jan 2012 17:30:40 +0000]
Added ContentReserve to the list of providers in insert_856_9
Ben Ostrowsky [Mon, 16 Jan 2012 18:25:29 +0000]
This time for sure!
Ben Ostrowsky [Mon, 16 Jan 2012 18:22:59 +0000]
Now handles subfield codes A-Z in addition to a-z (this is strictly speaking not allowed in data, but it exists anyway)
Ben Ostrowsky [Fri, 6 Jan 2012 17:32:30 +0000]
Added two related functions for updating a copy's owning_lib, both called
migration_tools.change_owning_lib. The difference is that if you call it with
the arguments (BIGINT, TEXT), it will try to look it up as a shortname and
pass the resulting actor.org_unit ID to change_owning_lib(BIGINT, INTEGER).
Ben Ostrowsky [Thu, 5 Jan 2012 14:11:06 +0000]
Avoid collisions while refreshing OPAC visibility
Ben Ostrowsky [Wed, 4 Jan 2012 19:39:39 +0000]
Replaced TRUNCATE with DELETE FROM in migration_tools.create_cards
Ben Ostrowsky [Thu, 1 Dec 2011 12:53:11 +0000]
Replaced TRUNCATE with DELETE FROM
Ben Ostrowsky [Mon, 21 Nov 2011 15:56:04 +0000]
Horizon-specific functions, beginning with a date conversion routine
Ben Ostrowsky [Tue, 18 Oct 2011 15:19:20 +0000]
Added new helper utility: migration_tools.refresh_opac_visible_copies
Ben Ostrowsky [Fri, 16 Sep 2011 13:39:42 +0000]
Avoiding collisions with existing permission groups and existing permissions
Ben Ostrowsky [Tue, 13 Sep 2011 18:20:58 +0000]
Improved check_ou_type
Ben Ostrowsky [Tue, 13 Sep 2011 17:55:19 +0000]
Added new function: check_ou_depth
Ben Ostrowsky [Fri, 9 Sep 2011 19:16:39 +0000]
insert_856_9: more accurate detection of incumbent subfield-9 data
Ben Ostrowsky [Mon, 8 Aug 2011 12:53:52 +0000]
Added migration_tools.zip_to_city_state_county
Ben Ostrowsky [Mon, 25 Jul 2011 16:28:50 +0000]
When Geonames has no data for a ZIP, use patron data even if --believegn
Galen Charlton [Fri, 22 Jul 2011 16:05:38 +0000]
fix bug introduced in r739
When replacing named captures for Perl 5.8 compatibility,
need to make sure regex groups that shouldn't capture
are marked as such.
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Wed, 20 Jul 2011 13:06:29 +0000]
Added --believegn flag to believe Geonames over patron data where they conflict
Ben Ostrowsky [Wed, 20 Jul 2011 13:05:17 +0000]
Added --belivegn flag to believe Geonames over patron data where they conflict
Ben Ostrowsky [Tue, 19 Jul 2011 18:15:29 +0000]
Handle percents in apply_circ_matrix
Ben Ostrowsky [Tue, 28 Jun 2011 12:09:05 +0000]
Added attempt_timestamptz
Ben Ostrowsky [Thu, 23 Jun 2011 17:10:17 +0000]
Utilities to nuke data
Ben Ostrowsky [Fri, 10 Jun 2011 17:07:40 +0000]
Munging ind2 in insert_856_9
Ben Ostrowsky [Wed, 11 May 2011 13:25:01 +0000]
New utility. Give it city-state-zip data in tab-separated format and it will give you the winning city/state for each zip in the same format.
Ben Ostrowsky [Tue, 10 May 2011 18:46:59 +0000]
Script by Galen, used in dedupe/merge processing.
Ben Ostrowsky [Thu, 5 May 2011 16:45:50 +0000]
Avoiding error message about non-numeric comparison; reporting on tags that are not strictly numeric
Ben Ostrowsky [Thu, 5 May 2011 16:33:18 +0000]
Call attention to subfields appearing more than once within a single tag
Ben Ostrowsky [Wed, 6 Apr 2011 13:48:09 +0000]
New function: migration_tools.change_call_number (copy_id BIGINT, new_label TEXT).
If the old or new label is ##URI##, the label will not be changed.
If the old label's volume is vacated by the change, it will be marked deleted.
Ben Ostrowsky [Wed, 23 Mar 2011 14:21:17 +0000]
Updated field names (recuring->recurring) in one function and added another to insert 856 $9 as needed
Ben Ostrowsky [Wed, 9 Mar 2011 18:11:33 +0000]
Routine to create cards in a migration schema
Ben Ostrowsky [Tue, 15 Feb 2011 13:08:36 +0000]
Function for assigning standing penalties
Ben Ostrowsky [Tue, 15 Feb 2011 13:07:48 +0000]
New options: --quiet, --ignoresubtitle
Ben Ostrowsky [Thu, 13 Jan 2011 16:27:18 +0000]
Adding metarecord spells as functions for convenience
Ben Ostrowsky [Mon, 22 Nov 2010 19:26:45 +0000]
Added --ignore-indexes option to grab more data, much of which will be junk to filter out downstream
Ben Ostrowsky [Thu, 18 Nov 2010 21:09:03 +0000]
A few adjustments
Ben Ostrowsky [Thu, 18 Nov 2010 20:33:38 +0000]
Handles items as well -- row length is a two-byte field
Ben Ostrowsky [Thu, 18 Nov 2010 19:29:55 +0000]
Works fine on TITLE.DB, but misses the boat on other files.
Ben Ostrowsky [Thu, 18 Nov 2010 16:10:20 +0000]
(cosmetic) Adjust digits in column numbers to fit data
Ben Ostrowsky [Thu, 18 Nov 2010 16:06:18 +0000]
Tools for handling Bibliofile *.DB/*.DBD files
Galen Charlton [Fri, 12 Nov 2010 15:18:55 +0000]
die if cannot open output or exception files
Signed-off-by: Galen Charlton <gmc@esilibrary.com>
Ben Ostrowsky [Mon, 8 Nov 2010 21:16:32 +0000]
A few refinements
Ben Ostrowsky [Mon, 8 Nov 2010 16:51:27 +0000]
A tool for generating Apache vhost configs from a table modeled after actor.org_unit_setting.
USAGE: ./opac_settings.pl --bootstrap /openils/conf/opensrf_core.xml --schema m_sage
Input will look like this:
evergreen=# select * from m_sage.opac_settings where org_unit = 107;
id | org_unit | name | value
-----+----------+-------------------+-----------------------------------
36 | 107 | opac.server_name | "conhs.sage.eou.edu"
93 | 107 | opac.server_alias | "www.conhs.sage.eou.edu"
147 | 107 | opac.directory | "condonhs"
251 | 107 | opac.ssl_cert | "ssl/wildcard.sage.eou.edu.crt"
301 | 107 | opac.ssl_key | "ssl/wildcard.sage.eou.edu.key"
Multiple libraries may share an 'opac.directory' (branches of a system with the same OPAC look and feel).
Multiple 'opac.server_alias' rows may be defined.
Ben Ostrowsky [Fri, 22 Oct 2010 16:28:37 +0000]
migration_tools.stage_not_applicable_asset_stat_cats
Ben Ostrowsky [Sun, 19 Sep 2010 01:11:42 +0000]
Better output-file renaming
Ben Ostrowsky [Fri, 10 Sep 2010 17:48:17 +0000]
Generates a TSV with egid and MARCXML snippets to preserve for post-deduping reinsertion (insert_tags).
Ben Ostrowsky [Fri, 10 Sep 2010 14:56:56 +0000]
Added feedback to remind the user where to find the results (since it doesn't go to STDOUT).
Ben Ostrowsky [Wed, 8 Sep 2010 16:07:48 +0000]
New function: apply_circ_matrix('m_foo.action_circulation').
Sets duration, fine, and max_fine names and values, and renewal_remaining,
based on a circ matrix (and patron profiles, and circ modifiers) already in place.
Ben Ostrowsky [Fri, 20 Aug 2010 16:27:50 +0000]
More precise deduping for insert_tags
Ben Ostrowsky [Fri, 20 Aug 2010 16:25:32 +0000]
Added function: insert_tags (marc, '<datafield tag="999" ... </datafield>')
Inserts into the proper numerical order, ignoring fields that are duplicates of each other.
Ben Ostrowsky [Thu, 10 Jun 2010 19:47:35 +0000]
attempt_phone can take '' as second argument to refrain from adding an area code; added is_blank(TEXT) for convenience
Ben Ostrowsky [Thu, 3 Jun 2010 15:44:09 +0000]
New function: migration_tools.is_staff_profile(actor.usr.profile), returns boolean
Ben Ostrowsky [Thu, 27 May 2010 15:15:24 +0000]
Built-in manual gave wrong command-line arguments. Fixed.
Ben Ostrowsky [Thu, 6 May 2010 16:29:04 +0000]
Added two MARC-manipulating functions from Galen
Ben Ostrowsky [Fri, 20 May 2011 19:52:18 +0000]
Eliminating one line of Perl 5.10 dependency
Ben Ostrowsky [Fri, 20 May 2011 16:16:41 +0000]
Fixed typo
Ben Ostrowsky [Fri, 20 May 2011 16:13:09 +0000]
Rewrote some regular expressions to make them compatible with Perl versions before 5.10
Ben Ostrowsky [Fri, 13 May 2011 15:54:46 +0000]
To be run immediately after normal EG installation