Matching Rules But Only The Latest Record Creates a Golden Record
F
Fathia Jama
started a topic
over 2 years ago
I am loading records for person (customer). We have some matching rules and so on. When loaded we see 6 records, as the file was loaded 6 times, and it should be matching....but only the latest record creates a Golden Record. Previously loaded versions of the record ALSO created one Master. So I think I should have 6 in the match group of Source records, or I should have multiple matches. What am I missing here?
Anna Rider The problem was wanting to keep values as they are in the very FIRST version of any record with that value non-NULL. I have solved the problem by using an Enricher, the Enricher looks ahead to the current Master record. If the value is not NULL in the Master it copies that version of the value to the Source record...retaining it indefinately.
I can see one record in the MD_ and one in the GD_ table (this is the LATEST, or 6th record, in each case).
When the 5th file was loaded the 5th record was in MD_ and GD_ table.
No records showing in the DU_ table for same source key.
So if this is the case, why don’t I see records 1-5 in the Match Group with the Golden Record loaded from file 6? These source records (once the 6th iteration of the file is loaded) do not produce Masters, get added to the match group, or create their own Master or Golden records. Shouldn’t they be in the match group or treated as singletons in MD_ or GD_ tables in that case?
This question seems very detailed so if we can't resolve it here, you may want to open a support ticket.
Without seeing sample data, I am not sure what your file looks like. Do your records all have the same PubID and SourceID? If so, then instead of creating new master records, xDM will simply identify the existing record and assume you want to update the record.
Therefore, the golden record's value may change based on the new update, but you would not see any new master records created or any new records in the DU table because you're loading an update, NOT a new master record. There's no matching to do.
Notice when I load the same Adele Mackson record with the same PublisherID and SourceID, it updates the golden to her new last name Adele Richardson. No matching occurred because the source record was fundamentally updated. There wasn't a NEW source record that would be a duplicate, which should be matching and merging.
Does this make sense? Does it answer your question?
Thanks for the info. I think this might be the case. The root issue is that I have a Survivorship Rule. This rule uses a loaded date (custom field added and populated by enruchment). The rule sorts ASC by the loaded date and persists 3 columns of data from the first record loaded. It is the same Pub and Source key so you are correct with that.
The problem is that the rule does not seem to get applied, because while we want to persist the first values for these columns, it probably does not "survive" those values because there is no matching...instead it updates them to the latest and does not use the rule. Any way to get this to work properly?
Brent Van Allen I don't follow the abstract description. Can you include sample data, the exact survivorship rule, the current result, and the expected/desired result so I can understand your use case?
Anna Rider The problem was wanting to keep values as they are in the very FIRST version of any record with that value non-NULL. I have solved the problem by using an Enricher, the Enricher looks ahead to the current Master record. If the value is not NULL in the Master it copies that version of the value to the Source record...retaining it indefinately.
Great. Sounds like you found a good solution that doesn't depend on the survivorship rule but does a copy via enricher to give you the intended end result. Well done.
Fathia Jama
I am loading records for person (customer). We have some matching rules and so on. When loaded we see 6 records, as the file was loaded 6 times, and it should be matching....but only the latest record creates a Golden Record. Previously loaded versions of the record ALSO created one Master. So I think I should have 6 in the match group of Source records, or I should have multiple matches. What am I missing here?
Brent Van Allen
Thank You.
BVA
- Oldest First
- Popular
- Newest First
Sorted by PopularFathia Jama
So if this is the case, why don’t I see records 1-5 in the Match Group with the Golden Record loaded from file 6? These source records (once the 6th iteration of the file is loaded) do not produce Masters, get added to the match group, or create their own Master or Golden records. Shouldn’t they be in the match group or treated as singletons in MD_ or GD_ tables in that case?
Fathia Jama
This question seems very detailed so if we can't resolve it here, you may want to open a support ticket.
Without seeing sample data, I am not sure what your file looks like. Do your records all have the same PubID and SourceID? If so, then instead of creating new master records, xDM will simply identify the existing record and assume you want to update the record.
Therefore, the golden record's value may change based on the new update, but you would not see any new master records created or any new records in the DU table because you're loading an update, NOT a new master record. There's no matching to do.
For example, check out this Customer B2C tutorial: https://www.semarchy.com/tutorials-content/customer-b2c-demo/#6
Go to the section "Browse updates"
Notice when I load the same Adele Mackson record with the same PublisherID and SourceID, it updates the golden to her new last name Adele Richardson. No matching occurred because the source record was fundamentally updated. There wasn't a NEW source record that would be a duplicate, which should be matching and merging.
Does this make sense? Does it answer your question?
Fathia Jama
Brent Van Allen
The problem is that the rule does not seem to get applied, because while we want to persist the first values for these columns, it probably does not "survive" those values because there is no matching...instead it updates them to the latest and does not use the rule. Any way to get this to work properly?
Fathia Jama
Anna Rider
Fathia Jama
Brent Van Allen
Thank You.
BVA
Fathia Jama
Anna Rider
-
Import Data Into Entities via Azure Data Lake
-
Recover Deleted(soft Delete) Record and Configure in Application
-
Data Quality in batch mode and real-time integration
-
Integration with analytics tools
-
Query/Load/Delete data with the REST API
-
Does the Done Tab in Inbox have a limit?
-
How Can I Trigger Enricher or Sql Procedure when deleting?
-
Unstructured and Semi Structured Data in Semarchy?
-
Read CSV files from AWS S3
See all 72 topics