Help › Data Cleanup

Data Cleanup — A Guide from Scratch

Over the years the same customer gets opened two or three times ("ABC Ltd. Şti." and "ABC LİMİTED ŞİRKETİ"), the same screw is entered under two different codes, cards are left without a tax number or a unit, and cards nobody uses any more crowd the lists. The result: the statement is split in two, stock shows up in two places, and an e-invoice bounces back because the tax number is missing. The Data Cleanup window finds duplicate card candidates, shows the two cards side by side and merges them with your approval; it lists missing fields; and it suggests deactivating cards that have not been used for a long time. No card is ever deleted, and every merge can be undone.

1. What this window is for

The window has four tabs: Duplicates (pairs of cards that may be the same), Missing fields (empty or invalid fields such as the tax number or the unit), Deactivation suggestions (cards no longer in use) and History (the merges done, and undoing them). Every tab works on the selected card type: the Account | Stock switch at the top right moves you between account cards and stock cards. It answers four questions:

• Which cards are actually the same? — the program looks at the tax ID, IBAN, phone, e-mail, barcode and name similarity and gives each pair a score from 0 to 100 (section 4).
• Which one stays, and what happens to the records? — you see the two cards side by side, choose the card that stays, and preview how many rows in which table will be moved before merging (sections 5 and 6).
• What is missing on which card? — missing or invalid fields are listed according to your country's rules; you fill them one by one or in bulk from the same tax ID (sections 8 and 9).
• I merged the wrong cards — now what? — every merge is stored with a detailed change log and is undone from the History tab (section 7).

2. Opening the window and permissions

The window is in the dock, inside the Management / HR tile: Data Cleanup. The top strip holds the tabs, each with the number waiting for the selected card type, and on the right the Account | Stock switch and the Scan and Refresh buttons. Right below it the "Last scan: …" line shows when the last scan ran; if there has never been one, it says "No scan has been run yet".

Permissions are ticked in the Data cleanup group on the user permissions screen in Settings. Every action that changes records (saving a scan, a decision, a fix, a merge) also requires the company's write gate to be open; read-only users see the lists but cannot start a saved scan either; when they try, they get the warning "A read-only user cannot change data cleanup records."

3. Scanning

The Scan button scans account and stock cards together and recalculates and saves the three lists: duplicate candidates, missing fields and deactivation suggestions. When it finishes, a green line gives the result for each type: "Account: 12 duplicate candidates, 340 cards with missing fields, 85 deactivation suggestions (4.2 s)". Inactive cards are not scanned; a card already merged or deactivated is not suggested again.

The scan is fast even in large companies because it does not compare every card with every other card: it only puts side by side the cards that share a clue (the same tax ID, IBAN, phone, e-mail, barcode, the same or a similar part of the name), and its queries use indexes instead of reading whole tables from start to end. A clue shared by very many cards (for example the same general e-mail address on dozens of cards) is left out of the comparison.

4. Duplicates: candidates and score

On the left of the Duplicates tab is the candidate list: Score, Card A, Card B (code and name) and Status. Above it are the "Search code or name" box, the status filter (default "Open (new + postponed)") and the score filter ("All scores", "At least 60 / 80 / 95 points"). The list starts from the highest score; 80 and above shows green, 60 and above yellow. Clicking a row opens the comparison of the two cards on the right (section 5).

How the score is calculated

Each pair's score is the sum of the reasons shown as coloured chips on the screen and stays between 0 and 100. The default threshold is 55: pairs with a lower score do not enter the list. A green chip is a strong reason (45 points or more), a blue chip a supporting reason, and a red chip a reason that lowers the score. For account cards:

For stock cards:

5. Comparison: which card stays

When you select a candidate, the right side shows the big score, the status and the reason chips at the top, and below them a field-by-field comparison of the two cards. With the selector in the column headers you choose the card that stays (the target): next to the chosen card a green Stays appears, next to the other a grey Will become inactive. By default Card A stays. Rows whose values differ are highlighted with a yellowish background. At the end of the table the Last movement, Balance, Open order, Barcodes and IBAN rows help you decide.

When choosing which card to keep, look at the following:

• Keep the code that fits your coding scheme and that the teams are used to — the code that becomes inactive can never be used again for a new card.
• Prefer the card with more and more recent movements; e-invoicing and bank matching are usually set up with that card.
• Keep the card whose tax number is correct and filled in; you can carry over missing fields from the other card (below).
• If the currencies of two account cards differ, the preview warns you; consult your accountant before merging two accounts in different currencies.

Carrying over fields

In writable fields whose value differs between the two cards, a selector appears before each value: you choose which value is written to the card that stays. If the field is empty on the card that stays and filled on the card that becomes inactive, the program selects the filled value by itself. The values you choose are written to the card that stays after the merge has finished, through the card window's own save path (with the same checks); below the table the note "After the merge, the N selected fields are written from the source card to the remaining card." appears.

6. Preview and merge

The preview does not run by itself: selecting a candidate only brings up the comparison. In the Preview: records that will change in the merge box below the comparison, pressing Preview counts the records that will change in the merge; the preview writes nothing. The Merge button becomes available after the preview; if you change the card that stays, the preview is cleared and you press Preview again. Its columns are Table, Column, Action, Rows (the number of rows linked to the source code) and Already on the target, with a Total at the bottom. The list of tables is not fixed: the program's register holds every known link (invoices, delivery notes, orders, quotes, receipts, cheques, account movements, line items, lots, recipes, production orders, price lists, contracts…), and the tables of new modules are recognised automatically from the names of their code columns.

The links that are moved also include: the lot origin and the recall supplier; the replenishment, sourcing, material declaration and FAI (first article inspection) supplier; stock card documents and supplier attachments (Shared Attachments); old production lines; and the stock and account file links in the file database (ek1). New tables opened after installation are also seen automatically from their code columns.

The Already on the target column shows conflicts: if the same row also exists on the target card (for example both cards are members of the same account group), the source row is not added a second time but dropped. Before it is removed, the dropped row is written in full to the merge's change log, and it comes back exactly on undo. If that log cannot be written, the whole merge is cancelled.

Identical rows are not doubled. If rows such as contact persons or addresses already exist exactly the same on the target, the source's copy is not added to the target a second time; it is dropped. A full copy of the dropped row is kept in the merge history and comes back on undo.

Credit risk setting. If both account cards have a credit risk setting, the most restrictive one stays on the target: for the behaviour the order on hold > block > send for approval > warn applies; for the temporary limit and the insurance limit the lowest amount is taken. The preview shows this as a warning line. On undo the target's setting from before the merge is written back exactly.

The unknown link warning

The program also looks for values equal to the source code in tables that are not in the register (a safety net). If it finds any, a yellow box appears in the preview: "The source code also appears in tables that are not in the register. These rows are not moved." The table, column and number of rows are listed below it. In this case the Yes, merge button stays disabled until you tick "I have checked; these rows stay on the source code." If a request is sent without the tick, the server rejects it too: "The source code also appears in tables that are not in the register; review the list and confirm explicitly."

Merge steps

1. Choose the card that stays and the field values to carry over, press Preview (Preview anyway in a large company) and read the preview.
2. If you wish, write a note of up to 300 characters in the "Note (optional)" box; the note is kept with the merge in the history.
3. Press Merge. A confirmation sentence appears: "All records of card SOURCE will be moved to card TARGET; SOURCE will become inactive. Do you confirm?"
4. Confirm with Yes, merge, or go back with Cancel.
5. The green result line gives the merge number and the number of rows moved: "Merged (#7): C0412 → C0098, 214 rows moved. The source card is inactive." The candidate moves to the Merged status; the open candidates of the inactivated card with other cards become Closed too (they reopen on undo).

The merge runs on the server as a single operation: the preview is rebuilt under a lock, rows are moved and summed, the balance is recalculated, the source card is marked inactive with its target recorded, and the change log and the access trail are written. If any step fails, nothing changes. A merge never deletes documents and never lets the source card's code be used again for a new card.

7. History and undo

The History tab lists every merge of the selected card type, newest first: # (merge number), Date, Source → target, Rows, Tables (rows per table), Who and Status (Applied or Undone; the name of the person who undid it is shown too).

Undo → Yes, undo: the change log is applied in reverse order. Moved rows go back to the source code, summed stock quantities are taken off the target and written back to the source, rows newly opened on the target are removed, rows dropped in a conflict are added back from the log, account balances are recalculated, and the source card becomes active again: "Merge #7 has been undone; the source card is active again." This button requires the card merge permission.

Partial undo

If some of the moved rows have changed since the merge (for example a moved invoice line was corrected or deleted), an exact undo is not possible: "Records have changed since the merge; the undo cannot be done exactly." In that case the button turns into Partial undo. A partial undo reverses everything that still matches the log, skips the rows that changed and tells you how many were skipped: "3 rows were skipped because they changed after the merge."

Safeguards

• Every row comes back by its identity. Each moved row is recognised by its row number or, if it has none, by a full copy of the row; rows with long texts such as addresses and notes are matched exactly too. If any row cannot be recognised, the undo stops and says "Records have changed since the merge; the undo cannot be done exactly." — it never reports a silent success. To continue, use Partial undo.
• The balance stays consistent with the movements. On undo, the account balance row of both cards is recalculated from the total of their account movements (no adding or subtracting); so even after a partial undo the balance matches the movements. For stock, a partial undo also recalculates the stock status from the movements.
• Undo does not depend on order. Independent merges are undone in any order you like: for example, an account merge can be undone without first undoing a stock merge.
• Settings and candidates come back. The target's credit risk setting from before the merge is written back, the other candidates closed by the merge reopen, and identical rows dropped in a conflict are added back from their copies in the history.

8. Missing fields

The Missing fields tab lists the empty or wrongly formatted fields on cards as card × field rows: Card, Field, Reason, Importance (High / Medium / Low) and Fix. Which field counts as required is decided not by the code but by your company's country package; the line "Country package: TR · 2,310 missing on 1,250 cards" at the top right shows which package is in use. The tiles at the top count the gaps by reason; clicking one filters the list to that field. The "All fields" selector and the "Search code or name" box filter too; the CSV and XLSX buttons download the list to a file.

The Türkiye package (TR)

• The tax number must be the 10-digit VKN for a legal entity and the 11-digit Turkish ID number for a private person (the e-invoice buyer identity); a value of any other length counts as "Invalid tax number".
• For the phone, it is enough for one of the four phone fields to be filled; the IBAN is searched for in the text of the Bank 1–3 fields.
• Barcode and minimum stock are only checked on cards of the stock family (not on expense and machine cards); a minimum stock of 0 counts as missing.

The Indonesia package (ID)

The Indonesia package has no tax-number or tax-office rule; instead it checks the identities that Coretax e-Faktur requires. The e-mail, phone, address and IBAN rules, and on the stock side the unit, VAT rate, product type, barcode and minimum stock rules, are the same as in Türkiye. The differences:

Under the NPWP rule, a card with a NIK does not count as missing (individual buyers). These four fields are kept in the tax section of the account and stock cards, from Statutory Obligations.

Fixing one by one

In the Fix column, editable fields show an input box and a Save button; if saving succeeds, "Saved" appears in its place. The value is written through the card window's own save path, so every check of the card window applies as is. The fields you can fix here: on the account card the tax number, tax office, e-mail, phone and address (NPWP and NITKU in Indonesia); on the stock card the VAT rate, product type and minimum stock (satuan and kode barang in Indonesia). For the unit, barcode and IBAN, go to the card itself with the Open card button. The value cannot be empty, is at most 200 characters and cannot contain a semicolon; the tax number format is checked too ("vergiNo format is invalid."). This requires the data cleanup permission (37).

9. Bulk filling from the same tax ID

The same company is often opened as several account cards (a branch, a delivery address, an old record); on one the tax office or address is filled, on the others it is empty. On account cards, when you choose a field on the Missing fields tab, the Fill from the same tax ID button appears. The program finds the cards that carry the same tax ID and have this field filled, and shows a suggestion table: Card, Suggested value, Source cards. If cards with the same tax ID carry different values, the row is marked with the Conflicting, not applied badge and skipped. If there is no suitable card, it says "No card with the same tax ID and this field filled was found."

Seeing the suggestion needs the data cleanup permission (37); the Apply suggestions button needs the card merge permission (38). When applying, the values are recalculated on the server (the value on the screen is not taken as given), at most 200 cards are written in one go, and each card is updated through the card window's own save path. At the end the message "Value written to N cards." appears and the list is refreshed; Close closes the suggestion table.

10. Deactivation suggestions

A card is suggested for deactivation if it meets all of the conditions below. The line "Criterion: no movement for 24 months, zero balance, no open document." at the top of the tab shows the period in force.

• No movement for a long time or Never had a movement — no movement at all in the last N months (24 by default).
• Zero balance — the account's balance is below 0.01; the stock card's company quantity on hand is 0.
• No open document — it has no open order; a stock card used as a component in a recipe also counts as "in use" and is not suggested.
• The card is older than N months — a newly opened card that has not yet had a movement is not suggested; a card that is already inactive is not suggested either.

The list has the columns Card, Last movement, Balance, Reason and Status; the status filter offers New, Approved, Rejected, Closed and All. A user with the card merge permission (38) chooses Deactivate or Keep active. A rejected card is not suggested again in later scans; a suggestion that no longer meets the conditions (for example because the card moved again) becomes Closed at the next scan.

11. The badge on the account and stock card

In the header of the account and stock card windows, the card's data cleanup status appears as a small badge. Clicking the badge opens the Data Cleanup window on the relevant tab:

• Inactive → TARGET (grey) — the card was merged into card TARGET. Hovering shows "This card was merged into card TARGET; use TARGET for new transactions."; clicking opens History.
• Inactive (grey) — the card was deactivated through a deactivation suggestion; clicking opens Deactivation suggestions.
• Possible duplicate: N (yellow) — the card has N duplicate candidates waiting for a decision; hovering shows the other cards' codes and scores, and clicking opens Duplicates with the highest-scoring candidate selected.

For a user without the data cleanup permission, or if the module's tables are not set up, the badge is not drawn at all; the card window is not affected.

12. On the phone, the weekly job, Pulse and the Owner Dashboard

On the phone (Garden + Step)

On a small screen the window opens in a three-step layout: 1. Card type (account cards or stock cards), 2. Duplicate candidates (score, Card A, Card B and status cards; a "Search code or name" box) and 3. Comparison (the score, the reason chips and the two cards' filled fields, last movement and balance). On the phone only the Keep separate and Postpone 30 days decisions are made (data cleanup permission, 37). Merging is done from the desktop window: the preview table and the field selection need a wide screen. Missing fields, deactivation suggestions and history are on the desktop too.

The weekly scan job

The server can also run the scan by itself once a week. The job is off by default; it is switched on when the system administrator writes VERI_TEMIZLIK_GOREV=1 in the server's .env file and restarts the Node service. The first round runs 10 minutes after start-up, the following ones once a week; it scans only the server's service company (HNR_FIRMA) and requires the company's write gate to be open and a service user with write rights to be defined. The job only produces suggestions: it never merges, never deactivates and never writes card fields. It shares the same lock as a manual scan (section 3).

Pulse and the Owner Dashboard

If the last scan left duplicate candidates waiting for a decision, or cards with a high-importance missing field, a single line appears in the Attention list of the Pulse interface and among today's warnings on the Owner Dashboard: "12 duplicate card candidates, missing fields on 340 cards." Tapping the line opens Data Cleanup on the Duplicates tab if there are candidates, otherwise on Missing fields. The numbers come from the last scan; the line is visible only to users with the data cleanup permission. It is an attention line and does not stop any work.

13. Warning messages and frequently asked questions

Related guides: Account Card · Stock Card · Statutory Obligations · Settings and Log · Owner Dashboard · Mobile.