Skip to content
Snippets Groups Projects

Skip duplicate records when importing

Merged Martin Burchell requested to merge ignore-duplicates into staging

Ideally we'd have unique IDs from the data source but we can't guarantee that so next best thing is to put a unique together validator for the body and timestamp fields. We do this at the REST API level as this is not possible for a text field in the database. Unfortunately because MySQL truncates microseconds we need to have a custom field in the serializer to truncate microseconds before validation. Shouldn't be a problem when we move to 5.6+ all round

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading