init
This commit is contained in:
252
README.md
Normal file
252
README.md
Normal file
@@ -0,0 +1,252 @@
|
||||
# Deployment Manager
|
||||
|
||||
A Kubernetes deployment manager that reads repository URLs from a database and automatically deploys them to Kubernetes clusters.
|
||||
|
||||
## Features
|
||||
|
||||
- **Automated Deployment**: Clone Git repositories, build Docker images, and deploy to Kubernetes
|
||||
- **Multi-Language Support**: Supports Node.js and Python applications
|
||||
- **Real-time Events**: Server-Sent Events (SSE) for real-time deployment status updates
|
||||
- **REST API**: Full REST API for managing repositories and deployments
|
||||
- **Worker Pool**: Concurrent deployment processing with configurable worker pool size
|
||||
- **Reconciliation Loop**: Automatic retry and status reconciliation
|
||||
- **Health Checks**: Built-in health check endpoints
|
||||
|
||||
## Architecture
|
||||
|
||||
The deployment manager consists of several components:
|
||||
|
||||
- **API Server**: HTTP server with REST endpoints and SSE support
|
||||
- **Worker Pool**: Concurrent workers that process deployment jobs
|
||||
- **Reconciler**: Background process that ensures desired state matches actual state
|
||||
- **Event Bus**: Pub/sub system for real-time event streaming
|
||||
- **Database**: SQLite database for repository metadata
|
||||
- **Executor**: Command execution and Docker operations
|
||||
- **Kubernetes Client**: Kubernetes resource management
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Go 1.21+
|
||||
- Docker
|
||||
- kubectl configured with cluster access
|
||||
- Git
|
||||
|
||||
### Building
|
||||
|
||||
```bash
|
||||
# Build the binary
|
||||
go build ./cmd/manager
|
||||
|
||||
# Or build with Docker
|
||||
docker build -t deployment-manager .
|
||||
```
|
||||
|
||||
### Running
|
||||
|
||||
```bash
|
||||
# Run locally
|
||||
./manager
|
||||
|
||||
# Run with Docker
|
||||
docker run -p 8080:8080 \
|
||||
-v /var/run/docker.sock:/var/run/docker.sock \
|
||||
-v ~/.kube:/root/.kube \
|
||||
deployment-manager
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `DB_PATH`: Path to SQLite database (default: `./manager.db`)
|
||||
- `MAX_WORKERS`: Number of concurrent workers (default: `2`)
|
||||
- `RECONCILE_TICK`: Reconciliation interval in seconds (default: `2`)
|
||||
- `HTTP_PORT`: HTTP server port (default: `8080`)
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Repositories
|
||||
|
||||
- `GET /api/repos` - List all repositories
|
||||
- `GET /api/repos?user_id=xxx` - List repositories by user
|
||||
- `POST /api/repos` - Create a new repository
|
||||
- `GET /api/repos/{id}` - Get repository details
|
||||
- `DELETE /api/repos/{id}` - Delete a repository
|
||||
- `POST /api/repos/{id}/stop` - Stop deployment
|
||||
- `POST /api/repos/{id}/restart` - Restart deployment
|
||||
|
||||
### Events
|
||||
|
||||
- `GET /events` - Server-Sent Events stream for real-time updates
|
||||
|
||||
### Health
|
||||
|
||||
- `GET /health` - Health check endpoint
|
||||
|
||||
## API Usage Examples
|
||||
|
||||
### Create a Repository
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/api/repos \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"repo_url": "https://github.com/user/my-node-app.git",
|
||||
"user_id": "user123",
|
||||
"type": "nodejs"
|
||||
}'
|
||||
```
|
||||
|
||||
### List Repositories
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/api/repos
|
||||
```
|
||||
|
||||
### Stream Events
|
||||
|
||||
```bash
|
||||
curl -N http://localhost:8080/events
|
||||
```
|
||||
|
||||
## Repository Types
|
||||
|
||||
### Node.js Applications
|
||||
|
||||
Expected structure:
|
||||
- `package.json` with dependencies
|
||||
- Entry point defined in `package.json` (default: `index.js`)
|
||||
- Exposes port 3000
|
||||
|
||||
### Python Applications
|
||||
|
||||
Expected structure:
|
||||
- `requirements.txt` with dependencies
|
||||
- `app.py` as entry point
|
||||
- Exposes port 8000
|
||||
|
||||
## Deployment Process
|
||||
|
||||
1. **Repository Creation**: Add repository via API
|
||||
2. **Job Queuing**: Reconciler detects `need_to_deploy` status
|
||||
3. **Cloning**: Worker clones the Git repository
|
||||
4. **Dockerfile Generation**: Auto-generates appropriate Dockerfile
|
||||
5. **Image Build**: Builds and pushes Docker image
|
||||
6. **Kubernetes Deployment**: Applies Kubernetes manifests
|
||||
7. **Status Update**: Updates repository status to `deployed`
|
||||
|
||||
## Kubernetes Resources
|
||||
|
||||
The deployment manager creates the following Kubernetes resources:
|
||||
|
||||
- **Deployment**: Manages pod replicas and updates
|
||||
- **Service**: Exposes the application internally
|
||||
- **ConfigMap**: Stores repository metadata
|
||||
- **Ingress**: External access (if ingress controller is available)
|
||||
|
||||
## Event Types
|
||||
|
||||
- `repo_created`: New repository added
|
||||
- `repo_updated`: Repository status changed
|
||||
- `repo_deleted`: Repository deleted
|
||||
- `deploy_started`: Deployment process started
|
||||
- `deploy_success`: Deployment completed successfully
|
||||
- `deploy_error`: Deployment failed
|
||||
- `log`: Real-time log messages
|
||||
|
||||
## Configuration
|
||||
|
||||
### Database Schema
|
||||
|
||||
The SQLite database contains a `repos` table with the following schema:
|
||||
|
||||
```sql
|
||||
CREATE TABLE repos (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
repo_url TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
user_id TEXT NOT NULL,
|
||||
type TEXT NOT NULL, -- nodejs | python
|
||||
image_tag TEXT,
|
||||
last_error TEXT,
|
||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
```
|
||||
|
||||
### Status Transitions
|
||||
|
||||
| From | To | Action |
|
||||
| -------------- | ---------- | -------------------- |
|
||||
| need_to_deploy | deploying | enqueue job |
|
||||
| deploying | deployed | success |
|
||||
| deploying | err | failure |
|
||||
| deployed | stopped | scale 0 |
|
||||
| stopped | restarting | scale up |
|
||||
| any | deleted | delete k8s resources |
|
||||
|
||||
## Development
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
deployment-manager/
|
||||
├── cmd/manager/ # Application entry point
|
||||
├── internal/
|
||||
│ ├── api/ # HTTP server and handlers
|
||||
│ ├── db/ # Database operations
|
||||
│ ├── events/ # Event bus and types
|
||||
│ ├── executor/ # Command execution and Docker
|
||||
│ ├── k8s/ # Kubernetes client
|
||||
│ ├── model/ # Data models
|
||||
│ ├── reconciler/ # Reconciliation logic
|
||||
│ └── worker/ # Deployment workers
|
||||
├── migrations/ # Database migrations
|
||||
├── manifests/ # K8s manifest templates
|
||||
└── Dockerfile
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
go test ./...
|
||||
```
|
||||
|
||||
### Building for Production
|
||||
|
||||
```bash
|
||||
# Build optimized binary
|
||||
CGO_ENABLED=1 GOOS=linux go build -a -installsuffix cgo -o manager ./cmd/manager
|
||||
|
||||
# Build Docker image
|
||||
docker build -t deployment-manager:latest .
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Docker socket not accessible**: Mount `/var/run/docker.sock` when running in Docker
|
||||
2. **kubectl not configured**: Ensure `~/.kube/config` is mounted and accessible
|
||||
3. **Database permissions**: Ensure the application can write to the database file
|
||||
4. **Port conflicts**: Change `HTTP_PORT` if 8080 is already in use
|
||||
|
||||
### Logs
|
||||
|
||||
Enable debug logging by setting the log level:
|
||||
|
||||
```bash
|
||||
LOG_LEVEL=debug ./manager
|
||||
```
|
||||
|
||||
### Health Checks
|
||||
|
||||
Monitor the deployment manager health:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/health
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License.
|
||||
Reference in New Issue
Block a user