migration-tools.git
6 years agomigration_tools.name_parse_out_last_first_middle_and_random_affix for LAST FIRST...
Jason Etheridge [Fri, 19 Feb 2016 02:01:09 +0000]
migration_tools.name_parse_out_last_first_middle_and_random_affix for LAST FIRST MIDDLE with no comma

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

6 years agomigration_tools.find_hold_matrix_matchpoint2
Jason Etheridge [Thu, 28 Jan 2016 17:08:58 +0000]
migration_tools.find_hold_matrix_matchpoint2

another convenience function

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoremove config fig.circ_matrix_circ_mod_test
Jason Etheridge [Thu, 24 Sep 2015 04:45:37 +0000]
remove config fig.circ_matrix_circ_mod_test

and _map from the list of base tables so that .build() will work with newer
versions of EG

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agomore money converters
Jason Etheridge [Mon, 10 Aug 2015 03:45:03 +0000]
more money converters

migration_tools.attempt_money_from_pennies
and migration_tools.attempt_money_from_pennies6

Avoids dividing by 100 when doing the conversion

2nd argument, the fail value, should not be pennies

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoduplicate_template_but_change_delay
Jason Etheridge [Thu, 6 Aug 2015 03:22:21 +0000]
duplicate_template_but_change_delay

Let's us get around duplicate key issues in some cases by tweaking the delay as
part of the cloning process.

Example:

    ubuntudeskfx=# SELECT * FROM migration_tools.duplicate_template(5,'{3,4}','00:30:00'::INTERVAL);

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoMinor typo fix in export_evergreen_library
Mike Rylander [Thu, 18 Jun 2015 13:49:27 +0000]
Minor typo fix in export_evergreen_library

Signed-off-by: Mike Rylander <mrylander@gmail.com>

7 years agotweak migration_tools.insert_856_9
Jason Etheridge [Mon, 20 Apr 2015 16:10:02 +0000]
tweak migration_tools.insert_856_9

rename original to migration_tools.insert_856_9_conditional, since it has
behavior based on 856$u content.  New version simply adds a $9 if one does
not already exist (regardless of org unit referenced)

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agohandle dates like 051980
Jason Etheridge [Tue, 24 Mar 2015 20:17:16 +0000]
handle dates like 051980

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoadd some in-db matchpoint tables, for potential ease of staging policy rules
Jason Etheridge [Wed, 18 Mar 2015 14:49:20 +0000]
add some in-db matchpoint tables, for potential ease of staging policy rules

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoconvenience wrappers for finding in-db rules
Jason Etheridge [Mon, 16 Mar 2015 18:22:57 +0000]
convenience wrappers for finding in-db rules

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agomake migration_tools.apply_circ_matrix_to_specific_circ use the more recent apply_cir...
Jason Etheridge [Mon, 16 Mar 2015 17:56:21 +0000]
make migration_tools.apply_circ_matrix_to_specific_circ use the more recent apply_circ_matrix code

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoadd a delimiter paramater, and a variant that returns an array of tags
Jason Etheridge [Wed, 25 Feb 2015 15:54:20 +0000]
add a delimiter paramater, and a variant that returns an array of tags

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years ago.get_marc_tag
Jason Etheridge [Sat, 21 Feb 2015 01:53:35 +0000]
.get_marc_tag

Useful since you can't get unnormalized values out of metabib.full_rec

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoMerge branch 'master' of git.esilibrary.com:migration-tools
Jason Etheridge [Thu, 5 Feb 2015 17:30:30 +0000]
Merge branch 'master' of git.esilibrary.com:migration-tools

7 years agomigration_tools.apply_circ_matrix_to_specific_circ
Jason Etheridge [Thu, 5 Feb 2015 17:30:26 +0000]
migration_tools.apply_circ_matrix_to_specific_circ

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoproc to clone a/t event definitions
Jason Etheridge [Tue, 3 Feb 2015 17:13:21 +0000]
proc to clone a/t event definitions

ubuntudeskfx=# SELECT * FROM migration_tools.duplicate_template(5,'{3,4}');
INFO:  created atevdef with id = 112
INFO:  created atevdef with id = 113
INFO:  -- UPDATE action_trigger.event_definition SET active = CASE WHEN id = 5 THEN FALSE ELSE TRUE END WHERE id in (5,112,113);
 duplicate_template
--------------------

(1 row)

ubuntudeskfx=# UPDATE action_trigger.event_definition SET active = CASE WHEN id = 5 THEN FALSE ELSE TRUE END WHERE id in (5,112,113);
UPDATE 3

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoexample script showing how to use 'fingerprinter' to deduplicate bib records
Galen Charlton [Thu, 29 Jan 2015 22:51:08 +0000]
example script showing how to use 'fingerprinter' to deduplicate bib records

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

7 years agolet's strip commas as well with attempt_money
Jason Etheridge [Wed, 14 Jan 2015 21:58:44 +0000]
let's strip commas as well with attempt_money

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agoan attempt_money method for a smaller numeric, given what we use for billings and...
Jason Etheridge [Wed, 14 Jan 2015 21:53:55 +0000]
an attempt_money method for a smaller numeric, given what we use for billings and payments

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

7 years agolet migration_tools.attempt_money strip dollar signs for us
Jason Etheridge [Wed, 14 Jan 2015 21:51:42 +0000]
let migration_tools.attempt_money strip dollar signs for us

8 years agoinclude affixes with _callnum_label_once_per_lib index
Jason Etheridge [Thu, 20 Nov 2014 22:16:30 +0000]
include affixes with _callnum_label_once_per_lib index

Signed-off-by: Jason Etheridge <jason@esilibrary.com>

8 years agoadd tables for call number affixes
Jason Etheridge [Thu, 20 Nov 2014 13:52:08 +0000]
add tables for call number affixes

8 years agoadd routine to calculate Code 39 mod 43 barcode checkdigits
Galen Charlton [Mon, 22 Sep 2014 18:44:03 +0000]
add routine to calculate Code 39 mod 43 barcode checkdigits

migration_tools.add_code39mod43_checkdigit($barcode);
If the source string is 13 or 14 characters long and contains only valid
Code 39 mod 43 characters, adds or replaces the 14th
character with a checkdigit computed according to the usual algorithm for library barcodes
using the Code 39 mod 43 symbology - see <http://en.wikipedia.org/wiki/Code_39#Code_39_mod_43>.  If the
input string does not meet those requirements, it is returned unchanged.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

8 years agoSimple script to export records belonging to a specific Evergreen OU and its descendents.
Galen Charlton [Thu, 28 Aug 2014 21:08:09 +0000]
Simple script to export records belonging to a specific Evergreen OU and its descendents.

Usage: export_evergreen_library shortname scratch_dir output_dir db_conn_params

For example:

$ mkdir inter out
$ ./export_evergreen_library SYS1 inter out '-U evergreen'

The script emits MARC bibs, undeleted patrons, all items for the OU, all loans
for the OU's patrons and items, current hold requests, and patron billings.

The script expects to be run on a box that can access the Evergreen database
via psql and an OpenSRF stack.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

8 years agofix construct that doesn't work in Perl 5.10
Galen Charlton [Fri, 1 Aug 2014 19:22:04 +0000]
fix construct that doesn't work in Perl 5.10

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

8 years agoimprove tool for cleaning up merge maps
Galen Charlton [Wed, 30 Jul 2014 20:53:47 +0000]
improve tool for cleaning up merge maps

The map cleaner/merger now correctly handles
cases where a given subordinate bib has a lead
bib that itself has a different lead bib.  It
also excludes cyclical loops in the merge map.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

8 years agomake the fingerprinter normalize OCLC control numbers
Galen Charlton [Wed, 30 Jul 2014 19:53:02 +0000]
make the fingerprinter normalize OCLC control numbers

Various ways of representing the same control number
are now normalized to "(OCoLC)" followed by the numeric
portion of the control number.

This patch also introduces a new module, Equinox::Migration::Utils,
which is meant for miscellaneous utility routines.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agoenhancements to patron/circ purge
Galen Charlton [Thu, 2 May 2013 16:13:17 +0000]
enhancements to patron/circ purge

- add a couple notes and instructions
- add a couple more tables to clear out
- use TRUNCATE where possible

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agoimprovements to bib nuking script
Galen Charlton [Thu, 2 May 2013 15:43:03 +0000]
improvements to bib nuking script

- truncate a few tables that don't have explicit FK
  links to bibs or copies
- nuke authority records
- inform the sysadmin of the next steps to take to
  complete the purge

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agodump_oracle_table_for_pg: add support for MARC8
Galen Charlton [Wed, 17 Apr 2013 00:52:33 +0000]
dump_oracle_table_for_pg: add support for MARC8

The dumper now groks --source-charset marc8

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agoNew tool: dump_oracle_query_output
Galen Charlton [Tue, 16 Apr 2013 22:42:30 +0000]
New tool: dump_oracle_query_output

Dumps results of query on Oracle database to file for loading into PostgreSQL

Usage: ./dump_oracle_query_output \
    [--sid oracle_sid] [--host oracle_host] --user oracle_user --pw oracle_password \
    --query sql_query \
    --out output_tsv_file [--help]

This exists because sqlplus lacks an easy way to dump arbitrary
query output to TSV or CSV.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agodump_oracle_table_for_pg: try to ensure dump output is UTF-8
Galen Charlton [Tue, 16 Apr 2013 21:02:25 +0000]
dump_oracle_table_for_pg: try to ensure dump output is UTF-8

Adds a --source-charset command-line parameter to specify
the source character encoding.  Note that CLOB/BLOB columns
are not considered (not that they were before this commit).

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agodump_oracle_table_for_pg: recognize CHAR columns
Galen Charlton [Tue, 16 Apr 2013 20:40:03 +0000]
dump_oracle_table_for_pg: recognize CHAR columns

Also don't complain if integer columns don't specify
a scale.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agowhen parsing state from city, anchor pattern on right
Galen Charlton [Fri, 8 Mar 2013 18:20:47 +0000]
when parsing state from city, anchor pattern on right

This avoids "ANCHOR POINT AK" from becoming {"ANCHORINT", "AK"}

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agomap Oracle NUMBER columns to correct Pg type
Galen Charlton [Wed, 6 Mar 2013 18:38:01 +0000]
map Oracle NUMBER columns to correct Pg type

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agonew utility: dump_oracle_table_for_pg
Galen Charlton [Wed, 6 Mar 2013 18:19:07 +0000]
new utility: dump_oracle_table_for_pg

This utility dumps the contents of an Oracle table
to file for loading into PostgreSQL.

Usage: dump_oracle_table_for_pg \
    [--sid oracle_sid] [--host oracle_host] --user oracle_user --pw oracle_password \
    --table oracle_table_name \
    --pg-table destination_pg_table_name \
    --out output_tsv_file --sql output_table_create_sql_file \
    [--column-prefix column_prefix] [--inherits-from base_pg_table] [--help]

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

9 years agorework attempt_cast function
Galen Charlton [Wed, 26 Dec 2012 17:00:32 +0000]
rework attempt_cast function

This function now returns the original value if it can
be cast to the desired type, and NULL otherwise.  For example:

UPDATE foo
SET price = migration_tools.attempt_cast(l_price, 'NUMERIC(8,2)')::NUMERIC(8,2);

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoimprove marc_parses() test
Galen Charlton [Fri, 19 Oct 2012 14:10:29 +0000]
improve marc_parses() test

If the Leader/09 is not 'a' and the record has diacritics,
MARC::Record and friends don't complain until you try to
emit a MARCXML record.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agostart work on a function for creating staff users in bulk
Galen Charlton [Wed, 3 Oct 2012 20:24:52 +0000]
start work on a function for creating staff users in bulk

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agonew function to set indicators in MARC fields
Galen Charlton [Mon, 10 Sep 2012 15:42:20 +0000]
new function to set indicators in MARC fields

migration_tools.set_leader(marc, tag, indicator_pos, value):

Set indicator value of a specified MARC field.
The first argument is a MARCXML string.
The second argument is a MARC tag.
The third argument is the indicator position, either 1 or 2.
The fourth argument is the character to set the indicator value to.
All occurences of the specified field will be changed.
The function returns the revised MARCXML string.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agocreate base staging tables for acquisitions
Galen Charlton [Thu, 30 Aug 2012 20:13:31 +0000]
create base staging tables for acquisitions

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agotypo fix
Galen Charlton [Wed, 29 Aug 2012 21:16:39 +0000]
typo fix

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoteach migration_tools.change_call_number() about label classes
Galen Charlton [Wed, 29 Aug 2012 21:01:54 +0000]
teach migration_tools.change_call_number() about label classes

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoMerge branch 'master' of git.esilibrary.com:migration-tools
Galen Charlton [Fri, 17 Aug 2012 20:11:34 +0000]
Merge branch 'master' of git.esilibrary.com:migration-tools

10 years agoremove last trace of vestigial bib modifier
Galen Charlton [Fri, 17 Aug 2012 20:10:25 +0000]
remove last trace of vestigial bib modifier

This modifier was removed in commit 8e63d0bdd; this commit
just removes the last reference.  Thanks to Justin Hopkins
for spotting this.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoswitch from action.find_circ_matrix_matchpoint to action.item_user_circ_test
Jason Etheridge [Fri, 17 Aug 2012 19:28:37 +0000]
switch from action.find_circ_matrix_matchpoint to action.item_user_circ_test

10 years agonew function - migration_tools.make_stub_bib()
Galen Charlton [Fri, 17 Aug 2012 19:13:21 +0000]
new function - migration_tools.make_stub_bib()

Simple function to create a stub MARCXML bib from a set of columns.
The first argument is an array of tag/subfield specifiers, e.g., ARRAY['001', '245a', '500a'].
The second argument is an array of text containing the values to plugin into each field.
If the value for a given field is NULL or the empty string, it is not inserted.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd author to 'edition' fingerprint
Galen Charlton [Wed, 15 Aug 2012 18:35:24 +0000]
add author to 'edition' fingerprint

Title and edition alone aren't sufficient; for example,
Meg Cabot and Stephanie Meyer both published the first
edition of books called Twilight in 2005.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd actor.usr_standing_penalty to default production list
Galen Charlton [Wed, 8 Aug 2012 20:03:03 +0000]
add actor.usr_standing_penalty to default production list

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agofor logging of field-level merges, look for record ID in both 901 and 903
Galen Charlton [Tue, 7 Aug 2012 18:42:42 +0000]
for logging of field-level merges, look for record ID in both 901 and 903

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoclean up a bit of noise
Galen Charlton [Tue, 7 Aug 2012 18:19:00 +0000]
clean up a bit of noise

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agomake --reverse option work
Galen Charlton [Tue, 7 Aug 2012 18:18:17 +0000]
make --reverse option work

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd --reverse option to extract_loadset
Galen Charlton [Tue, 7 Aug 2012 18:09:37 +0000]
add --reverse option to extract_loadset

When supplied, output will contain the subordinate bibs
rather than the lead bibs.  This can be useful for getting
the subordinate bibs in a separate file for field-level
merging.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd new parallel field modifier to extract_holdings
Galen Charlton [Tue, 7 Aug 2012 14:56:40 +0000]
add new parallel field modifier to extract_holdings

Some systems embed holdings data in pairs of fields with different tags.

For example, given a record like this:

852 $h235.45 ABC$p12345
852 $h456.79 DEF$p78990
961 $t2
961 $t3

and a mapping file using the parallel modifier:

call    852 h
barcode 852 p
mattype 961 t   m:parallel

extract_holdings will extract two item records:

l_call           l_barcode    l_mattype
--------------   ----------   ---------
245.45 ABC       12345        2
456.79 DEF       78990        3

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agosimple function to merge fields from one MARCXML record into another
Galen Charlton [Mon, 6 Aug 2012 19:50:05 +0000]
simple function to merge fields from one MARCXML record into another

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoanother typo fix
Galen Charlton [Mon, 6 Aug 2012 17:56:39 +0000]
another typo fix

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd pgt to config import/export
Galen Charlton [Mon, 6 Aug 2012 17:51:22 +0000]
add pgt to config import/export

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agofix typo
Galen Charlton [Mon, 6 Aug 2012 17:50:28 +0000]
fix typo

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd config import/export logic to add new rows for config.rule_*
Galen Charlton [Mon, 6 Aug 2012 16:37:39 +0000]
add config import/export logic to add new rows for config.rule_*

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd a couple more tables to the config export
Galen Charlton [Mon, 6 Aug 2012 15:55:23 +0000]
add a couple more tables to the config export

Note: import will not correctly handle any prefixes/suffixes
assigned to copy locations unless IDs happen to be in sync

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd the corresponding simple library config import function
Galen Charlton [Mon, 6 Aug 2012 15:50:15 +0000]
add the corresponding simple library config import function

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoadd another table to the configuration export
Galen Charlton [Mon, 6 Aug 2012 15:42:22 +0000]
add another table to the configuration export

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agostart working on functions to export and import library configuration
Galen Charlton [Mon, 6 Aug 2012 15:37:11 +0000]
start working on functions to export and import library configuration

This export function is starting out both limited and simple; among
other things it assumes that the OU IDs will be the same between
the source and target databases.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agotrim excessive trailing whitespace from subfield contents
Galen Charlton [Thu, 2 Aug 2012 15:28:42 +0000]
trim excessive trailing whitespace from subfield contents

If a subfield has too much (arbitrarily defined as at least 10)
of trailing whitespace, trim the whitespace.  This works around
a problem applying certain stylesheets (like the MARCXML-to-MODS
stylesheet) that use a recursive XSLT function to trim whitespace.

Note that only "excessive" whitespace is trimmed; some systems
emit subfields that contain semantically significant trailing
whitespace in certain fields.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agodon't crash if record is dumped because it's missing its -ot
Galen Charlton [Wed, 1 Aug 2012 18:49:34 +0000]
don't crash if record is dumped because it's missing its -ot

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoalso set grace period when applying circ policies
Galen Charlton [Wed, 1 Aug 2012 13:49:34 +0000]
also set grace period when applying circ policies

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agofix error message displayed if exception file cannot be opened
Galen Charlton [Mon, 30 Jul 2012 18:47:37 +0000]
fix error message displayed if exception file cannot be opened

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoquiet a couple warnings
Galen Charlton [Mon, 30 Jul 2012 17:48:51 +0000]
quiet a couple warnings

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoutility script to prepare a file of MARCXML records for fingerprinter
Galen Charlton [Mon, 30 Jul 2012 17:42:23 +0000]
utility script to prepare a file of MARCXML records for fingerprinter

Given a two-column tab-delimited text file contain bib IDs and MARCXML, produces
a MARCXML file with the bib IDs in 903 fields.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoimprove marc_cleanup's parsing of MARCXML files
Galen Charlton [Mon, 30 Jul 2012 17:32:35 +0000]
improve marc_cleanup's parsing of MARCXML files

In particular, ones produced by use of MARC::File::XML->as_xml_record(),
which inserts namespace attributes in the record element.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agouse PL/PerlU rather than PL/Perl
Galen Charlton [Tue, 24 Jul 2012 19:31:08 +0000]
use PL/PerlU rather than PL/Perl

The trusted version of PL/Perl is not enabled by
default if you follow the installation instructions
for recent versions of Evergreen.

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agosimple stored function to check parseability of MARCXML blobs
Galen Charlton [Tue, 24 Jul 2012 19:25:58 +0000]
simple stored function to check parseability of MARCXML blobs

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoVery rough tool to help parse serctl.data from a Symphony export
Lebbeous Fogle-Weekley [Fri, 20 Jul 2012 14:34:48 +0000]
Very rough tool to help parse serctl.data from a Symphony export

Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>

10 years agoadd more copyright and license statements
Galen Charlton [Mon, 16 Jul 2012 16:10:48 +0000]
add more copyright and license statements

Signed-off-by: Galen Charlton <gmc@esilibrary.com>

10 years agoAdd a header line to explain the fields
Ben Ostrowsky [Wed, 11 May 2011 15:55:25 +0000]
Add a header line to explain the fields

git-svn-id: svn://nox.esilibrary.com/migration-tools@734 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoSome city/state data has no comma
Ben Ostrowsky [Mon, 7 Mar 2011 18:15:48 +0000]
Some city/state data has no comma

git-svn-id: svn://nox.esilibrary.com/migration-tools@725 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoNow with address line2!
Ben Ostrowsky [Wed, 13 Oct 2010 15:44:28 +0000]
Now with address line2!

git-svn-id: svn://nox.esilibrary.com/migration-tools@711 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoBetter parsing of city/state
Ben Ostrowsky [Wed, 30 Jun 2010 13:56:10 +0000]
Better parsing of city/state

git-svn-id: svn://nox.esilibrary.com/migration-tools@703 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoDon't clobber user_category1 and user_category2
Ben Ostrowsky [Fri, 25 Jun 2010 15:37:37 +0000]
Don't clobber user_category1 and user_category2

git-svn-id: svn://nox.esilibrary.com/migration-tools@702 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoNow extracting .COUNTY. fields from addresses
Ben Ostrowsky [Thu, 10 Jun 2010 20:00:51 +0000]
Now extracting .COUNTY. fields from addresses

git-svn-id: svn://nox.esilibrary.com/migration-tools@700 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoReinserting a missing space that was causing trouble
Ben Ostrowsky [Wed, 9 Jun 2010 14:08:45 +0000]
Reinserting a missing space that was causing trouble

git-svn-id: svn://nox.esilibrary.com/migration-tools@698 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFixed a bug that improperly parsed 'III' as a suffix.
Ben Ostrowsky [Wed, 26 May 2010 15:03:48 +0000]
Fixed a bug that improperly parsed 'III' as a suffix.

git-svn-id: svn://nox.esilibrary.com/migration-tools@694 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFixed a bug that could prevent name parsing in some cases.
Ben Ostrowsky [Wed, 26 May 2010 15:00:05 +0000]
Fixed a bug that could prevent name parsing in some cases.

If the name includes an undotted abbreviation (Mrs, Jr) immediately followed by a comma,
the previous version was removing the comma, which got in the way of parsing that name.

git-svn-id: svn://nox.esilibrary.com/migration-tools@693 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFirst working draft of analyze_zips.sql
Ben Ostrowsky [Tue, 22 Dec 2009 14:28:43 +0000]
First working draft of analyze_zips.sql

git-svn-id: svn://nox.esilibrary.com/migration-tools@653 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoRegex fix was a bug; this undoes it
Ben Ostrowsky [Wed, 16 Dec 2009 21:30:45 +0000]
Regex fix was a bug; this undoes it

git-svn-id: svn://nox.esilibrary.com/migration-tools@651 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFixed regex to handle lines that begin with a dot and something other than a capital...
Ben Ostrowsky [Wed, 16 Dec 2009 21:02:26 +0000]
Fixed regex to handle lines that begin with a dot and something other than a capital letter (i.e., not a field label)

git-svn-id: svn://nox.esilibrary.com/migration-tools@650 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoMinor fix to a comment line
Ben Ostrowsky [Fri, 6 Nov 2009 16:51:51 +0000]
Minor fix to a comment line

git-svn-id: svn://nox.esilibrary.com/migration-tools@637 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoNew script to extract lost items from users.data, when present
Ben Ostrowsky [Fri, 6 Nov 2009 16:47:34 +0000]
New script to extract lost items from users.data, when present

git-svn-id: svn://nox.esilibrary.com/migration-tools@636 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoCorrectly parsing Unicorn patron names and addresses
Ben Ostrowsky [Fri, 6 Nov 2009 16:06:23 +0000]
Correctly parsing Unicorn patron names and addresses

git-svn-id: svn://nox.esilibrary.com/migration-tools@635 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoUnicorn charge.data to TSV parser
Ben Ostrowsky [Mon, 2 Nov 2009 21:12:03 +0000]
Unicorn charge.data to TSV parser

git-svn-id: svn://nox.esilibrary.com/migration-tools@634 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoUnicorn bill parser finished
Ben Ostrowsky [Mon, 2 Nov 2009 15:17:22 +0000]
Unicorn bill parser finished

git-svn-id: svn://nox.esilibrary.com/migration-tools@633 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoUnfinished draft of Unicorn bill converter
Ben Ostrowsky [Fri, 30 Oct 2009 19:40:47 +0000]
Unfinished draft of Unicorn bill converter

git-svn-id: svn://nox.esilibrary.com/migration-tools@632 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoUtter pedantry? Perhaps, but misleading comments are bugs too.
Ben Ostrowsky [Fri, 30 Oct 2009 18:55:39 +0000]
Utter pedantry? Perhaps, but misleading comments are bugs too.

git-svn-id: svn://nox.esilibrary.com/migration-tools@630 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFirst draft mark two (with syntax errors fixed)
Ben Ostrowsky [Tue, 27 Oct 2009 20:18:57 +0000]
First draft mark two (with syntax errors fixed)

git-svn-id: svn://nox.esilibrary.com/migration-tools@628 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFirst finished draft of new unicorn_patrons_to_tsv.pl
Ben Ostrowsky [Tue, 27 Oct 2009 20:10:40 +0000]
First finished draft of new unicorn_patrons_to_tsv.pl

git-svn-id: svn://nox.esilibrary.com/migration-tools@627 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoFinished the bit that processes the records. Next up: actual output.
Ben Ostrowsky [Tue, 27 Oct 2009 19:57:22 +0000]
Finished the bit that processes the records. Next up: actual output.

git-svn-id: svn://nox.esilibrary.com/migration-tools@626 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoReforking unicorn importer; this is an unfinished version of a patron importer
Ben Ostrowsky [Tue, 27 Oct 2009 17:39:05 +0000]
Reforking unicorn importer; this is an unfinished version of a patron importer

git-svn-id: svn://nox.esilibrary.com/migration-tools@624 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoCleaning up older copy of unicorn_to_tsv.pl
Ben Ostrowsky [Mon, 26 Oct 2009 18:11:29 +0000]
Cleaning up older copy of unicorn_to_tsv.pl

git-svn-id: svn://nox.esilibrary.com/migration-tools@623 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agoUnicorn TSV-maker can now handle bill.data and charge.data too.
Ben Ostrowsky [Mon, 26 Oct 2009 18:05:56 +0000]
Unicorn TSV-maker can now handle bill.data and charge.data too.

git-svn-id: svn://nox.esilibrary.com/migration-tools@621 eee7cc8d-164e-4af6-8e1b-092a69004917

10 years agochange \r\n to \n if they occur
Jason Etheridge [Wed, 9 Sep 2009 22:00:28 +0000]
change \r\n to \n if they occur

git-svn-id: svn://nox.esilibrary.com/migration-tools@617 eee7cc8d-164e-4af6-8e1b-092a69004917