Initial template import

2026-02-05 21:19:44 +00:00
commit 75e3a05d36
15 changed files with 1177 additions and 0 deletions
--- a/demo-process/docs/notes.md
+++ b/demo-process/docs/notes.md
@@ -0,0 +1,168 @@
+# Data Synchronization Process - Documentation
+
+## Overview
+
+This process implements a data synchronization workflow that fetches data from an external REST API, transforms and validates it, and writes it to a PostgreSQL database.
+
+## Process Flow
+
+1. **Fetch Data from API** - Retrieves customer data from the external REST API endpoint
+2. **Transform Data** - Maps external data fields to internal schema
+3. **Validate Data** - Applies quality rules and validation logic using DMN decisions
+4. **Write to Database** - Persists validated data to the PostgreSQL database
+
+## Integration Points
+
+### REST API Connector
+- **Location**: `../connectors/rest-api/`
+- **Configuration**: See `resources/mappings.yaml` for endpoint details
+- **Authentication**: Bearer token (configured via `API_TOKEN` environment variable)
+- **Endpoints**:
+  - GET `/api/v1/data` - Fetch customer records
+  - PUT `/api/v1/data/{id}` - Update customer record
+
+### Database Connector
+- **Location**: `../connectors/database/`
+- **Type**: PostgreSQL
+- **Schema**: See `../connectors/database/schema.sql`
+- **Tables**:
+  - `customers` - Main customer data table
+  - `sync_log` - Synchronization audit trail
+
+## Decision Logic (DMN)
+
+The validation decision table (`dmn/decisions.dmn.xml`) evaluates records based on:
+- **Record Type**: customer, transaction, etc.
+- **Quality Score**: 0-100 numeric quality metric
+- **Required Fields**: Presence of mandatory fields
+
+### Validation Outcomes
+- **VALID** (Score ≥ 80) → Process immediately
+- **WARNING** (Score 50-79) → Flag for manual review
+- **INVALID** (Score < 50 or missing required fields) → Reject
+
+## Case Management (CMMN)
+
+When synchronization issues occur, a case is created to manage the investigation and resolution:
+
+1. **Investigate Issue** - Manual task to analyze the problem
+2. **Check API Status** - Automated check of API availability
+3. **Check Database Status** - Automated check of database connectivity
+4. **Resolve Issue** - Manual remediation task
+5. **Retry Synchronization** - Automated retry of failed sync
+
+## Configuration
+
+### Environment Variables
+
+Required environment variables (see `../config/secrets.example.env`):
+
+```bash
+# API Configuration
+API_TOKEN=your-api-bearer-token
+API_BASE_URL=https://api.example.com
+
+# Database Configuration
+DB_HOST=localhost
+DB_PORT=5432
+DB_NAME=processgit
+DB_USER=sync_user
+DB_PASSWORD=secure_password
+```
+
+### Data Mapping
+
+Field mappings are defined in `resources/mappings.yaml`:
+
+| External Field | Internal Field | Type    |
+|---------------|----------------|---------|
+| external_id   | customerId     | string  |
+| full_name     | customerName   | string  |
+| contact_email | email          | string  |
+| created_date  | createdAt      | datetime|
+
+## Error Handling
+
+### API Failures
+- **Strategy**: Retry with exponential backoff
+- **Max Retries**: 3
+- **Notification**: Alert ops team
+
+### Database Failures
+- **Strategy**: Log and continue
+- **Notification**: Alert ops team
+
+### Validation Failures
+- **Strategy**: Reject record
+- **Log Level**: Warning
+
+## Performance Considerations
+
+- **API Timeout**: 30 seconds
+- **Database Connection Pool**: 2-10 connections
+- **Batch Size**: Recommended 100 records per sync
+- **Frequency**: Configurable (default: hourly)
+
+## Testing
+
+### Unit Tests
+- Validate data transformation logic
+- Test decision table rules
+- Verify error handling
+
+### Integration Tests
+- End-to-end API to database flow
+- Connection failure scenarios
+- Data validation edge cases
+
+### Performance Tests
+- Load test with 10,000+ records
+- Concurrent sync operations
+- Connection pool behavior
+
+## Monitoring
+
+### Key Metrics
+- Sync success rate
+- Average processing time per record
+- API response times
+- Database query performance
+- Error rates by type
+
+### Alerts
+- Sync failure rate > 5%
+- API response time > 10s
+- Database connection pool exhaustion
+- Validation rejection rate > 20%
+
+## Troubleshooting
+
+### Common Issues
+
+**Issue**: Sync fails with API timeout
+- **Cause**: API endpoint slow or unavailable
+- **Resolution**: Check API status, increase timeout, contact API provider
+
+**Issue**: Database write failures
+- **Cause**: Connection pool exhausted or schema mismatch
+- **Resolution**: Check pool configuration, verify schema matches mappings
+
+**Issue**: High validation rejection rate
+- **Cause**: Data quality issues at source
+- **Resolution**: Review validation rules, contact data provider
+
+## Change Log
+
+### Version 0.1.0 (Initial Release)
+- Implemented basic data sync workflow
+- Configured REST API and database connectors
+- Added DMN validation rules
+- Created CMMN case for issue management
+- Documented configuration and operations
+
+## Support
+
+For issues or questions:
+- **Primary Contact**: {{.RepoOwner}}@processgit.local
+- **Integration Team**: integration@processgit.local
+- **On-Call**: See `metadata/ownership.yaml`