deployment-manager/README.md

# Deployment Manager

A Kubernetes deployment manager that reads repository URLs from a database and automatically deploys them to Kubernetes clusters.

## Features

- **Automated Deployment**: Clone Git repositories, build Docker images, and deploy to Kubernetes
- **Multi-Language Support**: Supports Node.js and Python applications
- **Real-time Events**: Server-Sent Events (SSE) for real-time deployment status updates
- **REST API**: Full REST API for managing repositories and deployments
- **Worker Pool**: Concurrent deployment processing with configurable worker pool size
- **Reconciliation Loop**: Automatic retry and status reconciliation
- **Health Checks**: Built-in health check endpoints

## Architecture

The deployment manager consists of several components:

- **API Server**: HTTP server with REST endpoints and SSE support
- **Worker Pool**: Concurrent workers that process deployment jobs
- **Reconciler**: Background process that ensures desired state matches actual state
- **Event Bus**: Pub/sub system for real-time event streaming
- **Database**: SQLite database for repository metadata
- **Executor**: Command execution and Docker operations
- **Kubernetes Client**: Kubernetes resource management

## Quick Start

### Prerequisites

- Go 1.21+
- Docker
- kubectl configured with cluster access
- Git

### Building

```bash
# Build the binary
go build ./cmd/manager

# Or build with Docker
docker build -t deployment-manager .
```

### Running

```bash
# Run locally
./manager

# Run with Docker
docker run -p 8080:8080 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/.kube:/root/.kube \
  deployment-manager
```

### Environment Variables

- `DB_PATH`: Path to SQLite database (default: `./manager.db`)
- `MAX_WORKERS`: Number of concurrent workers (default: `2`)
- `RECONCILE_TICK`: Reconciliation interval in seconds (default: `2`)
- `HTTP_PORT`: HTTP server port (default: `8080`)

## API Endpoints

### Repositories

- `GET /api/repos` - List all repositories
- `GET /api/repos?user_id=xxx` - List repositories by user
- `POST /api/repos` - Create a new repository
- `GET /api/repos/{id}` - Get repository details
- `DELETE /api/repos/{id}` - Delete a repository
- `POST /api/repos/{id}/stop` - Stop deployment
- `POST /api/repos/{id}/restart` - Restart deployment

### Events

- `GET /events` - Server-Sent Events stream for real-time updates

### Health

- `GET /health` - Health check endpoint

## API Usage Examples

### Create a Repository

```bash
curl -X POST http://localhost:8080/api/repos \
  -H "Content-Type: application/json" \
  -d '{
    "repo_url": "https://github.com/user/my-node-app.git",
    "user_id": "user123",
    "type": "nodejs"
  }'
```

### List Repositories

```bash
curl http://localhost:8080/api/repos
```

### Stream Events

```bash
curl -N http://localhost:8080/events
```

## Repository Types

### Node.js Applications

Expected structure:
- `package.json` with dependencies
- Entry point defined in `package.json` (default: `index.js`)
- Exposes port 3000

### Python Applications

Expected structure:
- `requirements.txt` with dependencies
- `app.py` as entry point
- Exposes port 8000

## Deployment Process

1. **Repository Creation**: Add repository via API
2. **Job Queuing**: Reconciler detects `need_to_deploy` status
3. **Cloning**: Worker clones the Git repository
4. **Dockerfile Generation**: Auto-generates appropriate Dockerfile
5. **Image Build**: Builds and pushes Docker image
6. **Kubernetes Deployment**: Applies Kubernetes manifests
7. **Status Update**: Updates repository status to `deployed`

## Kubernetes Resources

The deployment manager creates the following Kubernetes resources:

- **Deployment**: Manages pod replicas and updates
- **Service**: Exposes the application internally
- **ConfigMap**: Stores repository metadata
- **Ingress**: External access (if ingress controller is available)

## Event Types

- `repo_created`: New repository added
- `repo_updated`: Repository status changed
- `repo_deleted`: Repository deleted
- `deploy_started`: Deployment process started
- `deploy_success`: Deployment completed successfully
- `deploy_error`: Deployment failed
- `log`: Real-time log messages

## Configuration

### Database Schema

The SQLite database contains a `repos` table with the following schema:

```sql
CREATE TABLE repos (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  repo_url TEXT NOT NULL,
  status TEXT NOT NULL,
  user_id TEXT NOT NULL,
  type TEXT NOT NULL, -- nodejs | python
  image_tag TEXT,
  last_error TEXT,
  created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
  updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### Status Transitions

| From           | To         | Action               |
| -------------- | ---------- | -------------------- |
| need_to_deploy | deploying  | enqueue job          |
| deploying      | deployed   | success              |
| deploying      | err        | failure              |
| deployed       | stopped    | scale 0              |
| stopped        | restarting | scale up             |
| any            | deleted    | delete k8s resources |

## Development

### Project Structure

```
deployment-manager/
├── cmd/manager/          # Application entry point
├── internal/
│   ├── api/             # HTTP server and handlers
│   ├── db/              # Database operations
│   ├── events/          # Event bus and types
│   ├── executor/        # Command execution and Docker
│   ├── k8s/             # Kubernetes client
│   ├── model/           # Data models
│   ├── reconciler/      # Reconciliation logic
│   └── worker/          # Deployment workers
├── migrations/          # Database migrations
├── manifests/           # K8s manifest templates
└── Dockerfile
```

### Running Tests

```bash
go test ./...
```

### Building for Production

```bash
# Build optimized binary
CGO_ENABLED=1 GOOS=linux go build -a -installsuffix cgo -o manager ./cmd/manager

# Build Docker image
docker build -t deployment-manager:latest .
```

## Troubleshooting

### Common Issues

1. **Docker socket not accessible**: Mount `/var/run/docker.sock` when running in Docker
2. **kubectl not configured**: Ensure `~/.kube/config` is mounted and accessible
3. **Database permissions**: Ensure the application can write to the database file
4. **Port conflicts**: Change `HTTP_PORT` if 8080 is already in use

### Logs

Enable debug logging by setting the log level:

```bash
LOG_LEVEL=debug ./manager
```

### Health Checks

Monitor the deployment manager health:

```bash
curl http://localhost:8080/health
```

## License

This project is licensed under the MIT License.