Metadata Stores Community News #18

• Wednesday 29th May - University of Western Sydney’s Enterprise Research Data Catalogue with Toby O'Hara;
• Wednesday 5th June - ReDBox Release 1.6.1. with Duncan Dickinson;
• Wednesday 12th June - University of Melbourne’s Research data Registry with Simon Porter;
• Recorded (video) presentations now available Claiming the Territory - La Trobe University's Metadata Store - with Roderick Sadler - 8th May 2013 and DataConnect - a tour of University of Adelaide's Metadata Store - with Cathy Miller - 15th May 2013

Cancelled: Data Surgery 22nd May 2013

Please note that there will be no Wednesday Data Surgery on the 22nd May 2013.

Metadata Stores Community News #16

1. Reminder - Data Surgeries/Clinics will resume May 8th 2013
2. Wednesday May 8th 2013 La Trobe University will be talking about the impact of their Metadata Stores project and the importance of Identifiers - details to follow
3. Strategies for Change Management with Carolyn Hulsbergen (17th April 2013) is now available on the ANDS YouTube Channel: http://www.youtube.com/watch?v=vm5YjXgDwuA
4. Vincula, the RMIT Metadata Stores Project as described by Alan Lo is also now available [press link]

Metadata Stores Community News #14

1. First to submit Final Report for a Metadata Stores project is the University of Queensland's UQ Data Collections Registry (UQ-DCR) project - congratulations to Project Manager, Hoylen Sue!
2. Data surgeries/clinics will go into recess for Easter but resume on the 10th April 2013 (usual time). On the 17th April 2013, we have tentatively booked Carolyn Hulsbergen to talk about Change Management at Deakin University;
3. Round table: Optional Deliverables and delays on Wednesday 10th April 2013 will be a discussion to clarify any flow-on effects from release delay to ReDBox 1.6;
4. For those that missed the terrific presentations by Marianne Brown, Jon Hurn & Lynn Davies, and Maude Francis, recordings were successful and should be available soon after Easter.

Approaching Wednesday morning events for your calendar

1. Wednesday 20th February 2013: No Data Clinic due to Adelaide ReDBox Community day and a half
2. Wednesday 27th February 2013: TIM Matching Workshop with Natasha Simons (details to follow)
3. Wednesday 6th March 2013: ReDBox Intensive with Duncan Dickinson (details to follow)

NLA/Trove - unexpected behaviours: Wed 13 Feb 10:00am (Melb Time)

Recent experiences with the NLA/Trove test environment have resulted in a working list of unexpected behaviours. Most need some form of explanation.

We'll be talking through this list on Wednesday 13th February 2013 10:00am (NSW, Vic, Tas) (9:00am Qld) (9:30am SA) (8:30 NT) (7:00am WA) via GoTo Meeting - log-in details below...(places limited to 26).

N.B. This is a working list - please contact Simon Pockley about any amendments or additions you would like to make.

1. Testing environment
   
   TIM Beta is the system that NLA use to check the accuracy of records before they are loaded into production. It isn’t a development system. There are limitations to the testing environment.
   
   a. The test NLA records cannot be deleted, so tests cannot be rerun and new people need to be created and used each time.
   b. NLA ID in TIM Beta resolves to Trove Production rather than Trove Test. This can prompt concern/confusion that valid NLA ID's might be incorrectly being ascribed to real researchers and then published on the web.
   
   Status - no action. The testing environment in TIM Beta does not impact on the production environment
   
   
2. Access to TIM
   
   a. Access to TIM is on an individual basis without a generic link to the institution.
   Status - work-around. Establish a convention for managed usernames and passwords, for example, “ANDS.tim.matching01”, “ANDS.tim.matching02”, etc. for multiple users. A single sign-on could be established, remembering that multiple users cannot login to TIM with the same user name and password simultaneously. Generic usernames and email addresses will allow usernames to be reassigned and the password rese
   b. Also having to create individual IDs for Trove and TIM (and making these the same for the test instances of each) is cumbersome compared to AAF access to Research Data Australia.
   
   Status – No action. Signup to Trove is open to any user of Trove, not just those with access to the Australian Access Federation (AAF) credentials. One username and password is used for both Trove and TIM (production and test).
   
   
3. Transparency of record status
   
   There’s no indication whether records have been auto-matched or gone into TIM for manual matching.
   
   Status - work-around. Checking which records are in TIM waiting for matching is a manual process.
   
   
4. Identifiers lose their type
   
   Identifiers lose the type indicated in the contributor party record e.g. "scopus". Even NLA supplied party identifiers become type="contributor’s ISIL code".  Any identifiers and the key supplied in a contributor party record to Trove become type="contributor’s ISIL code" in the Trove party record.
   Status – low impact - no action
   
   
5. Single element names not harvested
   
   RIF-CS schema guidelines state that <namePartType> is optional. The reason for variation in how names are presented is that GeoNetWork uses ISO 19115 and/or ANZLIC as its native schema which allows free text entry of names in a single element. RIF-CS party records type person that have origins in either ANZLIC or ISO19115 will be not be harvested by TROVE due to the lack of <namePartType>. Note that this issue impacts on Party person records only. Group records without <namePartType> will be ingested by Trove.
   Status – low impact - no action
   
   
6. NLA converts keys to RDA URLS
   
   Not all institutions maintain or even create public profile pages. These are used as Source URLS in a Trove record.
   
   Status - work-around for where there are no source URLs for staff profiles is to use an optional function in Trove. If option is set to ‘on’ the entityID (Source URL) for a university's records in Trove are transformed to ANDS URLS. Trove uses a predefined URL structure to convert keys to a link to RDA and not to the researcher’s web page on the university web site. If set to ‘off’ Trove will use the source URL provided.
   
   For ReDBox/Mint users:
   
   Care should be taken that the source URL does not resolve to MINT which requires a logon, and not to a local party record as might be expected. Apply the workaround indicated above, as required.
   
   
7. Vanishing records
   
   As reported by UQ:  A feed of party records was harvested but some of the records neither appeared in Trove nor in TIM - they vanished without a trace. The harvest operation did not fail, since other party records from the same harvest of the feed appeared in TIM or were automatically matched and appeared in Trove.
   Status - behaviour yet to be replicated
   
   
8. Broken links when Researchers leave an institution (tombstone records)
   
   There is a need for Policy development regarding Researchers who move or even die. NLA has no control over contributor’s source links. Some universities will use the links to the researchers' local profile as the source URL. However, these links may not be maintained after the researcher leaves the universities. So in the long run there will be a lot of broken links in Trove.
   
   NLA comment: There is no need for a University to delete the record for a researcher when they move away from that institution. The NLA party identity is a container record and so can contain records from different universities for the same researcher. This is where some descriptive information would be valuable in the record e.g. “Left University of ZX in 2011”.
   
   Status – low impact - no action
   
   
9. Inconsistent matching algorithms
   
   Testing Trove's automatic matching algorithms are behaving unexpectedly:
   
   Issue first raised by Hoylen Sue at UQ - as follows
   
   Situation #1: party records are harvested for the first time. That is, Trove does not contain any party records with these RIF-CS keys.
   
   Situation #2:  when these unchanged party records are re-harvested. That is, Trove already has existing party records with these same RIF-CS keys. In this situation UQ has found the two party records for Watson and Drebber pass "Matching Rules Part A" and replace the existing party records -- which is the correct behaviour. But the party record for Holmes does not match and goes into TIM Beta -- which is not correct. It should have been automatically matched just like the other two. Neither us nor NLA have figured out why Holmes is treated differently.
   
   NLA comment:
   
   Paul has checked this problem as much as he can and when he sends the record to TIM manually it matches. Essentially the way he's doing it is exactly the way the harvester would send the records to TIM so we have no idea why the records aren't matching when we harvest them. Paul says we need more data to work with and he's reluctant to use your data as then you won't be able to use it. Perhaps if we had lots more test records to work with we might be able to see a pattern. Sorry I don't have a solution or a reason for the records not matching.
   
   Status - behaviour yet to be replicated
   
   
10. Complex matching rules
    
    There is an assumption that if local party records contain the NLA identifier as an identifier, then matching will occur automatically, but this is not necessarily the case. The matching rules are complex and deliberately conservative. Records which fail Part A of the matching rules then go to Part B where the NLA identifier is just one of the match points.
    
    Rules for matching party records: https://wiki.nla.gov.au/download/attachments/24379936/ARDCPIPMatchingRulesSpec-+Ver+2.0++20+Jan+2012.doc?version=1&modificationDate=1333064788000
    
    From the matching rules:
    
    The purpose of Part A: to decide if two records are actually the same record
    An incoming record is checked to determine if it has the same contributor ISIL and record ID as a record in Trove.
    
    If it does, a sanity check is applied. If the incoming record passes the sanity check, it is automatically matched and overlays (replaces) the matching record in Trove. If the incoming record fails the matching rules in Part A then matching rules in Part B are applied.
    
    The purpose of Part B: to decide if a record should be part of an existing identity.
    
    The rules specified in Part B are applied to an incoming record that has failed the matching rules specified in Part A.
    
    An incoming record is checked to determine if it has an NLA persistent identifier. If it does, a sanity check is applied and must be passed for a match to occur.
    
    Incoming records that fail the matching rules specified in Part A and B are passed by the identity service to the unmatched record queue for human review for matching or new record creation in TIM
    
    Example:
    
    A university creates a record for “Davies, Peter” containing the NLA party identifier for the existing Trove record for “Davies, Peter Eric” (the Trove record was contributed by Libraries Australia, ISIL code AU-ANL:PEAU).
    
    Result: Part A matching rules apply, but neither ISIL code or recordID match so Part B rules are applied.
    
    In the Part B matching rules the system looks at the incoming record for the NLA party identifier and the NLA’s ISIL (AU-ANL:PEAU) and the NLA party identifier, recorded in control/sources/source/objectXMLWrap.
    
    If the university added the NLA party identifier for “Davies, Peter Eric” and the NLA agency code to the incoming record, the system would run the sanity check. It would match on Davies, on Peter, but it wouldn’t find Eric in the incoming record and so the name would fail the matching rules. Because there is always the risk that a mistake could be made when the NLA party identifier is manually entered into another organisation’s record, the system has to be certain it’s the same person so it will always check all name parts and all name parts have to match.
    
    Result: In this example, if the names didn’t automatically match then the record failed the sanity checks at some point.
    
    Status –investigate. The example above explains why records containing an NLA party identifier will not necessarily match even though they have the same NLA Identifer.
    
    
11. RIF-CS 1.4 mapping to EAC-CPF (Encoded archival context for corporate bodies, persons and families)
    
    There is currently no separate overarching application profile document for NLA use of EAC-CPF. The NLA maintain mappings from MARC and RIF but these are both in flux at present as the NLA make changes to support RDA and RIF-CS 1.4.The mapping for RIF-CS 1.4 is finished but NLA’s IT is still to finish doing the style sheet to convert the records.
    
    Status – in progress
    
    
12. Name sequence or name order
    
    The order in which names are entered is important because the Trove System has its own way of processing first name and last name. Best to enter in natural order i.e. first name: Jane, last name: Smith
    
    Status – work-around
    

---------------------------------------------------
1. Please join my meeting.
https://www4.gotomeeting.com/join/902515079

2. Use your microphone and speakers (VoIP) - a headset is recommended.

Or, call in using your telephone.

Dial +61 2 8355 1031
Access Code: 902-515-079
Audio PIN: Shown after joining the meeting

Meeting ID: 902-515-079

Wednesday: Amir Aryani will answer your Trove questions (Coming) What happens when you've built your Metadata Store?

1. Amir Aryani will be available to answer and NLA/Trove questions following Hoyken Sue's presentation last week.

Wednesday 23rd January 2013 10:00am (NSW, Vic, Tas) (9:00am Qld) (9:30am SA) (8:30 NT) (7:00am WA) via GoTo Meeting - details below


2. Coming - Natasha Simons will be discussing how the team at Griffith University overcame challenges associated with their Research Hub.

Wednesday 30th January 2013 10:00am (NSW, Vic, Tas) (9:00am Qld) (9:30am SA) (8:30 NT) (7:00am WA) via GoTo Meeting - details below

This discussion will include:

1. what to do with dirty data?
2. getting support
getting access to data from different systems
3. finding new sources of data to input into your metadata store (e.g. our theses database)
4. what happens when your metadata store solution starts to compete with your publications repository?
5. developing a sustainability plan to get long-term support for the Hub post-project completion that might be useful to share.

1. Please join my meeting.
https://www4.gotomeeting.com/join/902515079

2. Use your microphone and speakers (VoIP) - a headset is recommended. Or, call in using your telephone.

Dial +61 2 8355 1031
Access Code: 902-515-079
Audio PIN: Shown after joining the meeting

Meeting ID: 902-515-079

Reminder Drop-in GoTo Meeting - Wednesday 5th Dec 10:00am (Melb Time)

Metadata Stores regular clinic/surgery - Wednesday 5th Dec 10:00am (Melb Time.

Meeting link: https://www4.gotomeeting.com/join/902515079

Or, call in using your telephone.

Dial +61 2 8355 1031

NEW Access Code: 902-515-079

Audio PIN: Shown after joining the meeting

If you have already downloaded the client

NEW GoTo Meeting ID: 902-515-079

Reminder Drop-in GoTo Meeting - Wednesday 28th Nov 10:00am (Melb Time)

Metadata Stores regular clinic/surgery - Wednesday 28th Nov 10:00am (Melb Time.

Meeting link: https://www4.gotomeeting.com/join/902515079

Or, call in using your telephone.

Dial +61 2 8355 1031

NEW Access Code: 902-515-079

Audio PIN: Shown after joining the meeting

If you have already downloaded the client

NEW GoTo Meeting ID: 902-515-079

Reminder Data Surgery Clinic (tomorrow) Wednesday 7th November

We will resuming our weekly data surgeries/clinics tomorrow, Wednesday 7th November at 10:00am. For anyone new to these drop-in sessions, they are very useful if you are interested in what others are doing and/or being introduced to people who might be able to help with particular problems.

Simply log into GoTo meeting at 10:00am

Meeting Link: https://www4.gotomeeting.com/join/435051343
Meeting ID: 435-051-343
Support information is here - http://ands.org.au/events/webinars/onlinemeetings.html
Or, call in using your telephone. Dial +61 2 8355 1030 Access Code: 435-051-343
Audio PIN: Shown after joining the meeting

Best wishes

Simon

Next weekly data surgery will be on the 7th November 2012

Please note that due to eResearch in Sydney next week and a clash this week, the next Metadata Stores weekly data clinic/surgery will be held at 10:00am on Wednesday the 7th November 2012.

For those in Queensland, there will be a Queensland Roundtable on the 25 Oct 2012 10:00-12:00pm local Brisbane Time.

Draft Agenda

Current Project Status - Each project
Current national level activity update from ANDS - Simon Pockley
Other business

If you would like to log-in by phone then contact Andrew White (ANDS/QCIF Research Data Analyst)

M: 0468 989 096
T: +61 7 3735 6735
F : +61 (0)7 3735 6650
Email: andrew.white@ands.org.au

Reminder: Metadata Stores - Clinic/Surgery: Wednesday 10:00am-11:00am

If you would like to continue the discussion about Data Management Planning tools begun by Toby O'Hara...then drop into today's Metadata Stores - Clinic/Surgery: Wednesday 10:00am-11:00am

by following the GoTo meeting link:

Meeting Location: https://www4.gotomeeting.com/join/435051343

Use your microphone and speakers (VoIP) - a multimedia headset is recommended. Or, call in using your telephone.

Dial +61 2 8355 1030
Access Code: 435-051-343
Audio PIN: Shown after joining the meeting
Meeting ID: 435-051-343

Press for Information about GoTo Meeting technical requirements etc .

This is intended to be a weekly catch-up. An opportunity to raise issues, share and compare useful tips and resources.

Reminder: Metadata Stores - Clinic/Surgery (Wednesday)

Metadata Stores - Clinic/Surgery: Wednesday 10:00am-11:00am

Drop-in to this regular event by following the GoTo meeting link:

Meeting Location: https://www4.gotomeeting.com/join/435051343

Use your microphone and speakers (VoIP) - a multimedia headset is recommended. Or, call in using your telephone.

Dial +61 2 8355 1030
Access Code: 435-051-343
Audio PIN: Shown after joining the meeting
Meeting ID: 435-051-343

Press for Information about GoTo Meeting technical requirements etc .

This is intended to be a weekly catch-up. An opportunity to raise issues, share and compare useful tips and resources.