Content Manager User Discussions
cancel

Migrate documents from Centera document store

Regular Contributor.. gerardj Regular Contributor..
Regular Contributor..

Migrate documents from Centera document store

We need to migrate over 8 TB of records from an EMC Centera document store to an ISILON document store in  Content Manager 9.1.2.1123.  We tried the "Transfer Documents" through the application.  It failed instantly with the error below:

"Processing Records  (uri=15599) ... HPE Content Manager Workgroup Server on 'WGVIP' reported an error. [CENTERA:FPClip_Exists]: FP_OPERATION_NOT_ALLOWED - The use of this operation is restricted (transid='WGSERVER4/16114/EXISTS_CLIP')"

So, we are using the PowerShell/SDK script below, however it is very slow (approx. 30 GB a day).  Has anyone performed this activity before who can offer any suggestions to improve performance?

Thanks!

# define log file
$logPath = "d:\Migrate_Centera_to_ISILON_LP.log"

#Add-Type -Path "D:\Program Files\Hewlett Packard Enterprise\Content Manager\HP.HPTRIM.SDK.dll"
Add-Type -Path "D:\Hewlett-Packard\HP Records Manager\HP.HPTRIM.SDK.dll"

#$LocalStoreName = "Main Document Store"
$LocalStoreName = "Archive Document Store"
$ISOLONStoreName = "ISILON Document Store"
$SearchString = "store:$($LocalStoreName)"
$Database = New-Object HP.HPTRIM.SDK.Database
$Database.id = 'XX'
$Database.workgroupservername = "WG_VIP"
$Database.Connect()

#fetch the store and exit if missing
$Tier1Store = $Database.FindTrimObjectByName([HP.HPTRIM.SDK.BaseObjectTypes]::ElectronicStore, $ISOLONStoreName)
if ( $Tier1Store -eq $null ) {
    Write-Error "Unable to find store named '$($ISOLONStoreName)'"
    exit
}

#search for records eligible for transfer
$Records = New-Object HP.HPTRIM.SDK.TrimMainObjectSearch -ArgumentList $Database, Record
$Records.SearchString = $SearchString

$foundRecords = $Records.Count
Write-Host "Found $foundRecords records"
$x = 0

#transfer each record
foreach ( $Result in $Records ) 
{
    $Record = [HP.HPTRIM.SDK.Record]$Result
    $record.TransferStorage($Tier1Store, $true)
    Write-Host "Record $($Record.Number) transfered"
    $timestamp = get-date -Format "yyyy-MM-dd HH:mm:ss"
    Add-Content $logPath -value ("$timestamp`t$($Record.Number) transferred") 
    $x++
    Write-Progress "$x of $foundrecords" -PercentComplete ($x/$foundRecords * 100)
}

$Database.Disconnect()

 

 

 

 

3 REPLIES
Sten28 Outstanding Contributor.
Outstanding Contributor.

Re: Migrate documents from Centera document store

In the past I've used DataPort and SQL extracts to faciliate record and document/revision transfers between datasets. Possibly a cleaner/faster approach; I managed 500,000 electronic objects per day on average.

 

Regular Contributor.. gerardj Regular Contributor..
Regular Contributor..

Re: Migrate documents from Centera document store

Hi Sten28,

I've considered using Dataport, but in one dataset we have 22 million records on Centera.  It would be a considerable task splitting that into manageable exports and then importing them.  I am hoping for a solution that doesn't make me lose my will to live.  I believe Dataport only exports the latest revision? 

Do you (or does anyone) know if the Transfer Documents option in the product works?  It consistently fails on the first record, even with "correct any errors" checked..  Still waiting on MF Support for an answer on that one.

Thanks

Sten28 Outstanding Contributor.
Outstanding Contributor.

Re: Migrate documents from Centera document store

22 million is quite a bit, but not impossible to split up. I've done about 8 million records myself at most.

You are correct in saying that DataPort only gets the latest revision, however this is where the "SQL" part comes in - I've used SQL scripts to grab metadata for where the electronic objects (and revisions) are, and then used that in an import file to re-injest into a new dataset without having to 'export' objects at all.

By 'updating on duplicate', DataPort will then injest records and their revisions with no issue.

I'm not so familiar with the Transfer Documents feature myself, unless someone else on these forums provides any insight, you'll need to wait and see what MF come back with.