Skip to content

Add tabular preprocessing - #1094

Open
ahzero7d1 wants to merge 19 commits into
mainfrom
tabular-preprocessing
Open

ahzero7d1 wants to merge 19 commits into
mainfrom
tabular-preprocessing

Conversation

@ahzero7d1

@ahzero7d1 ahzero7d1 commented Mar 30, 2026 •

Copy link
Copy Markdown
Collaborator

Initial implementation of tabular data preprocessing

Closes #1062, fixes #1063

Changes

  • Add validation on the web app side to reject datasets containing: nan values, empty strings
  • Implement standardization for numerical features
  • Store standardization metadata as part of model serialization

TODO

  • Add one-hot-encoding for non-numerical features

@ahzero7d1 ahzero7d1 changed the title Tabular preprocessing Add tabular preprocessing Mar 31, 2026
@JulienVig
JulienVig self-requested a review September 11, 2026 15:21

@JulienVig JulienVig left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing things required before merging (for me to handle):

  • Use metadata statistics during model validation
  • Current titanic sample csv is rejected by webapp because of missing data
  • Standardization is done locally. This can lead to different weight scales between clients. Should the standardization statistics be averaged by the server?
  • Unit tests for new tabular preprocessing functions
  • Validate CSV right away when connecting it
  • update docs/examples/custom_task.ts
  • throw error if categorical features have missing data instead of creating a new category

Comment thread discojs/src/processing/tabular.ts Outdated
Comment thread discojs/src/processing/tabular.ts Outdated
Comment thread discojs/src/training/disco.ts Outdated
Comment thread discojs/src/training/disco.ts Outdated
Comment thread discojs/src/processing/index.ts
Comment thread discojs/src/serialization/model.ts Outdated
@JulienVig
JulienVig force-pushed the tabular-preprocessing branch from 2370ab2 to 06fb7e5 Compare September 24, 2026 13:29
@JulienVig
JulienVig force-pushed the tabular-preprocessing branch from ab1e113 to 33ed710 Compare September 28, 2026 13:20
@JulienVig
JulienVig force-pushed the tabular-preprocessing branch from 33ed710 to fc042ee Compare September 28, 2026 13:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tabular data NaN processing tabular data preprocessing improvements

2 participants