DSEDataSourceExplorer
DSE 1.0.1

User manual

This guide explains the normal end-to-end operating flow of Data Source Explorer (DSE).

Use it when you want one detailed reference that covers:

1. What the application is designed to do

DSE is an operator-oriented desktop application for discovering data sources, inspecting their structure, preparing extraction or mapping definitions, and generating reusable output artifacts.

In practical terms, the usual workflow is:

  1. choose a target and run Discover
  2. inspect the discovered source tree
  3. review or edit connection information
  4. test connectivity
  5. load tables / collections / buckets / other objects
  6. review the object structure and data preview
  7. build a schema catalog / data catalog view of what matters
  8. compose output targets when you need mapped destination objects
  9. generate deployables for one or more runtime targets
  10. review generated files and, when needed, generate the graphical schema

2. Main window layout

The main window is divided into several working zones.

Top command area

Use this area to:

Left navigation tree

The left side shows the discovered estate. Depending on the environment, you may see:

Use the tree to move from broad scope to a specific source or object.

Right work area

The right side shows:

3. Start a session

A normal session starts like this:

  1. Launch the application.
  2. Select the UI language if needed.
  3. In the target box, enter the machine, host, environment, or discovery path that you want to inspect.
  4. Choose the discovery kind if the application offers multiple target modes.
  5. Click Discover.

Good practice

4. Discover and explore sources

After discovery finishes, review the left tree carefully.

What to look for:

Recommended operator flow

  1. expand the tree level by level
  2. select one source of interest
  3. review the summary card on the right
  4. decide whether to open details, edit connection, remove the scope, or continue into object loading

5. Review details and edit connection information

For the selected source, DSE can expose actions such as:

Open details

Use Open details when you want a read-only technical summary of the selected node or connection.

Edit connection

Use Edit connection when:

Recommended sequence:

  1. edit only the fields you need
  2. save the connection changes
  3. immediately run Test connection
  4. only then continue to object loading or deployable generation

6. Test connection

Use Test connection before any heavy operation.

A good connection test confirms that:

If a test fails:

7. Load objects and inspect structure

After the connection succeeds, click Load tables or the equivalent source-loading command.

Depending on source type, this may load:

How to inspect objects

  1. select an object in the object list
  2. review Table info or the equivalent summary tab
  3. open Preview to inspect sample rows or documents
  4. use Row details when you need to inspect a selected record more closely
  5. review Catalog Schema and Data Catalog when you are turning the source structure into a reusable catalog definition

8. Build the Schema Catalog and Data Catalog

The catalog-related tabs are where DSE becomes more than a browser.

Schema Catalog

Use this area to define the structural view of the source objects that matter for extraction or downstream deployment.

Typical uses:

Data Catalog

Use this area to enrich or review the selected fields and objects from a more user-facing or reusable perspective.

Typical uses:

Good practice

9. Generate Output Data directly

Use Output Data when you want to produce data artifacts from the selected catalog.

This is useful for:

Typical output types include:

Before a large run:

10. Compose output targets

Use Compose targets when you need destination-oriented mappings instead of raw source browsing.

In the target composer you normally work with three layers:

  1. available source rows/fields on the left
  2. one or more destination targets in the middle
  3. objects and field mappings on the right

Typical mapping workflow

  1. create a destination target
  2. choose the target preset or system type
  3. create one or more target objects
  4. select source rows/fields
  5. add them to the selected target object
  6. review the mapping grid
  7. resolve validation warnings
  8. save the target definition

Use cases

Validation tips

Fix validation issues before saving, especially:

11. Generate deployables

Use Generate deployables to produce a portable runner or related deployment artifacts from the catalog and/or saved Output Targets.

Input modes

DSE supports two common patterns:

With saved targets enabled, use the dropdown and + button to build the job. The chips across the top are logical target definitions, not necessarily separate physical destinations.

Destination route: consolidate several targets

Enable Route selected targets to one destination when several logical targets should be written into one physical destination.

For example:

MongoDB + Couchbase + InfluxDB -> MongoDB

DSE creates one destination object per selected target by default. In the routing table, the Destination object column is directly editable. Use clear, unique names and verify that each row reports mapped fields.

Universal Destination Writers

Every current Output Target preset now has a generated-runner writer path. The capability badge shows the implementation technology rather than forcing unsupported destinations into a Plan-only state:

This is a single canonical-record architecture: source records are mapped once and then translated by the selected destination adapter. The physical destination therefore does not need to match the source technology or the logical Output Target technology.

For Salesforce, Dynamics 365 CRM, SharePoint, ServiceNow, Workday and SAP OData, the direct API writer intentionally exposes Append only. Destructive full-object Replace requires a reviewed custom gateway policy.

Append vs Replace

Destination instance section

For an executable route, this section configures where the deployable will connect and write.

Review fields such as:

Use Test connection before generation when possible.

For production, prefer keeping the destination connection outside the generated schema and set DSX_DESTINATION_CONNECTION_STRING in the runner environment.

AI Advisor handoff

The AI Advisor can recommend a destination and send a proposed route to Generate Deployables. This remains approval-gated: AI analysis does not itself write to an external database. Review the route, capability badge, object names, write mode, and connection before generating/running it.

Platforms and runtimes

Choose one or more deployment platforms, for example:

Then choose one or more compatible runtime identifiers, such as:

Output options

CSV and Excel remain available as audit/review outputs even when a physical destination route is used. Also review:

Deployment readiness panel

Before generating, read the preflight/readiness section. It reports missing platform/RID/input/output values and route problems such as an unregistered custom destination, blank/duplicate destination objects, unsupported write modes, or missing mapped fields.

Context help

Use the Routing help button in the Destination route card, or press F1 while the Generate Deployables window is active, to open the detailed deployables help topic.

Generate

When the configuration is ready:

  1. verify the selected target chips;
  2. verify the destination capability and editable object names;
  3. verify Append/Replace and the destination connection;
  4. verify platform/runtime/output choices;
  5. click Generate.

12. Job configuration persistence

When you leave the Generate deployables page, the application can ask whether you want to keep the job configuration.

If you choose to keep it

The next time you open the page, DSE restores the saved configuration, including:

If you choose not to keep it

The temporary job configuration is cleared.

Recommendation

Keep the job only when you intend to come back and continue the same deployment setup.

13. Review generated deployables

After generation, DSE opens a generated-deployables summary area.

Use this view to:

14. Review deployable run results

The results window is the post-run validation surface.

It usually provides:

Recommended review workflow

  1. open the results window
  2. select each generated file at least once
  3. inspect sheet tabs or text previews
  4. verify row counts and column names
  5. open the physical file when you need external validation

15. Generate the graphical schema

From the results window, use Generate schema to build a visual graph of the run.

The graph can connect:

This is useful for:

While the graph is being built, the progress indicator should show activity.

Save graph

Use Save graph to export the schema in one of the supported formats, such as:

16. Workspaces

Use the Workspace menu to manage reusable sessions.

Common actions include:

Recommended practice

17. Plugins, logs, and Help

Plugins

Open Plugins to review the providers/extensions available to the application. If a source type is missing or behaves differently than expected, check the plugin state first.

Logs

Open Logs whenever you need technical detail about discovery, connection, generation, or runtime problems.

Help

Open Help for searchable documentation. Press F1 for context help.

18. Suggested operating sequence for a clean production-style run

  1. open or create a workspace
  2. discover the intended scope
  3. select a source and review its details
  4. edit connection information if needed
  5. test the connection
  6. load objects
  7. inspect preview/data structure
  8. build the schema/data catalog selection
  9. compose targets if the run requires explicit destination mapping
  10. open generate deployables
  11. choose saved targets or catalog input
  12. review destination settings, platforms, runtimes, output formats, and folder
  13. generate the deployable
  14. open the summary and then the results
  15. review files and previews
  16. generate and save the graphical schema if documentation is needed
  17. save the workspace

19. Troubleshooting checklist

Discovery returns too little or too much

Connection test fails

Objects do not appear after loading

Deployable generation is blocked

Results look incomplete

20. Final recommendations

For the most reliable experience:

Enterprise reliability, lineage and CDC

Generated deployables can enable Run IDs, structured lineage, integrity hashes, transient retries, a schema-scoped run lease/heartbeat and Append-mode resume checkpoints. These features strengthen restart safety and governance for migration jobs.

They do not convert snapshot reads into log-based CDC. Production CDC requires source-specific transaction-log/change-feed adapters and durable provider cursors. See Docs/Enterprise-Reliability-CDC-Governance-HA-Plan.md for the implementation and certification roadmap.

← PreviousNext →