Loading...

Data entities that survive DMF refresh: the F&O checklist

Custom data entities look trivial on paper and cause outsized production impact when small configuration details are missed. This is the checklist teams apply before a custom entity leaves a sprint.

Data entities that survive DMF refresh: the F&O checklist

Custom data entities are where F&O talks to the rest of the enterprise stack. They power DMF imports and exports, OData endpoints, dual-write maps, and recurring integrations. When one is built carelessly, the integration breaks at the worst possible moment - usually after a sandbox refresh when the test team isn't watching.

Seven things in a custom entity decide whether it survives the first DMF refresh and the first production import.

1. Staging table comes with the entity

Every public entity that supports DMF imports needs its own staging table. The staging table is the intermediate storage DMF uses to hold incoming data before pushing it into the target entity. An entity marked isPublic = Yes with no staging table fails DMF imports with errors about missing intermediate storage.

Create the staging table as an extension of EntityName_EntityStagingGeneric with matching fields, then link it via the entity's StagingTable property. Step one for any integration-capable entity.

2. Field groups match the data contract

Field groups control what appears in DMF templates and OData projections. Fields not in a group are effectively invisible to consumers even though they compile fine. For a customer master entity, typical groups are:

  • Keys - AccountNum, DataAreaId
  • Identification - Name, NameAlias, CustGroup
  • Financial - PaymTermId, CurrencyCode, CreditLimit
  • Address - PrimaryAddressLocation, PrimaryAddressRoles
  • Extension - custom fields added by the project

DMF template generation reads these groups. Grouping is what produces self-documenting import templates for the business-side data team.

3. Mandatory fields reflect business reality, not schema

Making a field Mandatory = Yes on an entity because the underlying table column has a NOT NULL constraint misunderstands what the property means. Entity Mandatory is about what the business operation requires, not what the database allows.

The classic failure: two-phase data loads where a record is created in phase 1 and enriched in phase 2. If the enrichment field is marked Mandatory on the entity, phase-1 imports fail - even though the underlying table tolerates the temporary null.

Set mandatory based on what the business process actually requires at import time, not what the database accepts.

4. Composite entities for header-line transactions

Transactions that need header + lines imported atomically (sales orders, purchase orders, journals) require composite entities. Wrap two regular entities with a parent-child relationship and configure:

  • Line entity: IsParentIdentificationKey = Yes
  • Line entity: own staging table
  • Line entity: AllowCreate = Yes when line-level inserts are expected

Composite entities are one of F&O's less-documented features. Microsoft's pattern exists but teams routinely miss it until their first order-import integration fails mid-batch.

5. Change tracking for recurring integration

Without change tracking, recurring integrations pull full snapshots every cycle, drowning downstream systems in duplicates. Enabling Change Tracking (Enable Track Changes = Yes) means consumers requesting $deltatoken in OData queries receive only records modified since the last call.

Change tracking is the difference between a 10-minute daily sync and a 4-hour daily sync.

6. DMF mapping preservation across sandbox refresh

DMF database refreshes can reset environment-specific mappings. Teams running active integrations against a sandbox that gets refreshed without preparation discover their recurring integration stops working the Monday after.

Standard practice before refresh:

  • Export DMF projects from the target environment as a package
  • Document integration-user DMF permissions
  • Back up the DMFDataEntityDefinitionTable customizations if any
  • Post-refresh, reimport the DMF package and verify mappings

Microsoft has improved preservation logic in recent platform updates, but scripting the backup is still the way teams protect against regressions.

7. Integration-specific service account

Every integration should run as a dedicated service account with a scoped role (custom IntegrationRole extending IntegrationServicePrincipalRole), not as an admin. Logs then correctly attribute actions, credential rotation doesn't break audit, and the blast radius of a compromised credential is bounded.

Audit teams notice when everything in the log is attributed to "admin". Dedicated integration accounts are what keep the audit findings short.

Checklist for sign-off

An entity ready for production:

  • Its own staging table with matching field list
  • Field groups mapped to the business contract
  • Mandatory flags reflecting business rules, not schema
  • Composite wrapping when it's header-line
  • Change tracking enabled for recurring-integration use
  • DMF project exported and version-controlled with the model
  • Dedicated integration user with scoped role

The discipline feels tedious on the first entity and saves days of debug time on the twentieth.

Contact Us Now

Share Your Story

We build trust by delivering what we promise – the first time and every time!

We'd love to hear your vision. Our IT experts will reach out to you during business hours to discuss making it happen.

WHY CHOOSE US

"Collaborate, Elevate, Celebrate where Associates - Create Project Excellence"

SapotaCorp beyond the IT industry standard, we are

  • Certificated
  • Assured quality
  • Extra maintenance

Tell us about your project

close