Overview
Spree provides a comprehensive bulk data import and export system for managing large datasets. The system supports CSV file processing with configurable field mapping, asynchronous processing via background jobs, and real-time progress tracking in the admin interface.Import/Export System Diagram
Architecture
The import/export system uses several design patterns:- Single Table Inheritance (STI): Import and Export types inherit from base classes
- State Machine: Imports progress through states (pending → mapping → processing → completed)
- Schema Definition: ImportSchema classes define expected fields and validation
- Row Processors: Transform CSV rows into database records
- Event-Driven Processing: Background jobs handle heavy lifting asynchronously
- Registry Pattern: Types registered in
Spree.import_typesandSpree.export_types
Exports
Exports generate CSV files from filtered database records.Built-in Export Types
| Type | Description | Multi-line |
|---|---|---|
Spree::Exports::Products | Products with all variants | Yes |
Spree::Exports::Orders | Orders with line items | Yes |
Spree::Exports::Customers | Customer accounts | No |
Spree::Exports::GiftCards | Gift cards | No |
Spree::Exports::NewsletterSubscribers | Newsletter subscribers | No |
Export Model
The baseSpree::Export class provides:
Creating a Custom Exporter
Step 1: Create the Export Classapp/models/spree/exports/subscriptions.rb
to_csv Method to Your Model
app/models/spree/subscription.rb
config/initializers/spree.rb
config/locales/en.yml
Multi-line Exports
For exports where each record produces multiple rows (like products with variants):app/models/spree/exports/orders_with_items.rb
app/models/spree/order.rb
Export Filtering
Exports support Ransack filtering viasearch_params:
Imports
Imports process CSV files to create or update database records.Built-in Import Types
| Type | Description |
|---|---|
Spree::Imports::Products | Products and variants |
Import Workflow
Import Components
Import Model
Import Schema
Defines expected CSV fields:Import Mapping
Maps CSV columns to schema fields:Import Row
Represents a single CSV row:Row Processor
Transforms row data into database records:Creating a Custom Importer
Step 1: Create the Import Classapp/models/spree/imports/subscriptions.rb
app/models/spree/import_schemas/subscriptions.rb
app/services/spree/imports/row_processors/subscription.rb
config/initializers/spree.rb
config/locales/en.yml
Products Import Schema
The built-in products import supports these fields: Required Fields:slug- Product URL slugsku- Variant SKUname- Product nameprice- Variant price
status- Product status (active/draft/archived)description- Product descriptionmeta_title,meta_description,meta_keywords- SEO metadatatags- Product tagscompare_at_price- Original price for sale displaycurrency- Price currencywidth,height,depth,dimensions_unit- Dimensionsweight,weight_unit- Weightavailable_on,discontinue_on- Availability datestrack_inventory- Enable inventory trackinginventory_count,inventory_backorderable- Stock settingstax_category,shipping_category- Category assignmentsimage1_src,image2_src,image3_src- Image URLsoption1_name,option1_valuethroughoption3_name,option3_value- Variant optionscategory1,category2,category3- Taxon assignments (format: “Taxonomy -> Taxon -> Child Taxon”)
Handling Multi-Variant Products
The products import handles variants intelligently:- Master variant rows (no option values): Create/update the product and its master variant
- Non-master variant rows (with option values): Create additional variants for an existing product
Metafield Support
Both imports and exports support metafields dynamically: Export: Metafield definitions are automatically added as CSV columns using the formatmetafield.{namespace}.{key}.
Import: Map CSV columns to metafield definitions. The system automatically detects columns matching the metafield pattern and updates the corresponding metafield values.
Background Jobs
CreateRowsJob
Parses CSV and creates ImportRow records:ProcessRowsJob
Processes pending rows through row processors:GenerateJob (Exports)
Generates CSV files for exports:Events
Import Events
| Event | Trigger |
|---|---|
import.created | Import record created |
import.completed | All rows processed |
Export Events
| Event | Trigger |
|---|---|
export.created | Export record created (triggers generation) |
Import Row Events
| Event | Trigger |
|---|---|
import_row.completed | Row processed successfully |
import_row.failed | Row processing failed |
Configuration
Queue Configuration
config/initializers/spree.rb
Preferences
Imports support configurable delimiter:Key Files Reference
| File | Purpose |
|---|---|
core/app/models/spree/import.rb | Base import model |
core/app/models/spree/export.rb | Base export model |
core/app/models/spree/import_schema.rb | Base schema class |
core/app/models/spree/import_mapping.rb | Field mapping model |
core/app/models/spree/import_row.rb | Row model with processing |
core/app/models/spree/imports/products.rb | Products import type |
core/app/models/spree/import_schemas/products.rb | Products schema |
core/app/services/spree/imports/row_processors/base.rb | Base processor |
core/app/services/spree/imports/row_processors/product_variant.rb | Products processor |
core/app/models/spree/exports/products.rb | Products export |
core/app/models/spree/exports/orders.rb | Orders export |
core/app/models/spree/exports/customers.rb | Customers export |
core/app/jobs/spree/imports/create_rows_job.rb | Row creation job |
core/app/jobs/spree/imports/process_rows_job.rb | Row processing job |
core/app/jobs/spree/exports/generate_job.rb | Export generation job |
admin/app/controllers/spree/admin/imports_controller.rb | Admin imports controller |
admin/app/controllers/spree/admin/exports_controller.rb | Admin exports controller |
Permissions
Access is controlled via CanCanCan:current_ability ensuring users only export data they have access to.
