153 lines
3.2 KiB
Markdown
153 lines
3.2 KiB
Markdown
# 🚀 Quick Start Guide
|
|
|
|
Get your sitemap generator running in 3 steps!
|
|
|
|
## Step 1: Install Go
|
|
|
|
If you don't have Go installed:
|
|
- Download from https://golang.org/dl/
|
|
- Install Go 1.21 or later
|
|
- Verify: `go version`
|
|
|
|
## Step 2: Run the Application
|
|
|
|
### Option A: Using the run script (easiest)
|
|
```bash
|
|
cd sitemap-api
|
|
./run.sh
|
|
```
|
|
|
|
### Option B: Using Make
|
|
```bash
|
|
cd sitemap-api
|
|
make run
|
|
```
|
|
|
|
### Option C: Manual
|
|
```bash
|
|
cd sitemap-api
|
|
go mod download
|
|
go build -o sitemap-api .
|
|
./sitemap-api
|
|
```
|
|
|
|
## Step 3: Use the Application
|
|
|
|
1. **Open your browser** → http://localhost:8080
|
|
|
|
2. **Enter a URL** → e.g., `https://example.com`
|
|
|
|
3. **Set crawl depth** → 1-5 (default: 3)
|
|
|
|
4. **Click "Generate Sitemap"** → Watch real-time progress!
|
|
|
|
5. **Download XML** → Click the download button when complete
|
|
|
|
## Testing Multiple Users
|
|
|
|
Open multiple browser tabs to http://localhost:8080 and start different crawls simultaneously. Each will have its own UUID and progress stream!
|
|
|
|
## API Usage Examples
|
|
|
|
### Start a crawl
|
|
```bash
|
|
curl -X POST http://localhost:8080/generate-sitemap-xml \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"url": "https://example.com", "max_depth": 3}'
|
|
```
|
|
|
|
Response:
|
|
```json
|
|
{
|
|
"uuid": "550e8400-e29b-41d4-a716-446655440000",
|
|
"site_id": 123,
|
|
"status": "processing",
|
|
"stream_url": "/stream/550e8400-e29b-41d4-a716-446655440000",
|
|
"message": "Sitemap generation started"
|
|
}
|
|
```
|
|
|
|
### Monitor progress (SSE)
|
|
```bash
|
|
curl http://localhost:8080/stream/550e8400-e29b-41d4-a716-446655440000
|
|
```
|
|
|
|
### Download sitemap
|
|
```bash
|
|
curl http://localhost:8080/download/550e8400-e29b-41d4-a716-446655440000 -o sitemap.xml
|
|
```
|
|
|
|
### List all sitemaps
|
|
```bash
|
|
curl http://localhost:8080/sites
|
|
```
|
|
|
|
### Delete a sitemap
|
|
```bash
|
|
curl -X DELETE http://localhost:8080/sites/123
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Port already in use
|
|
```bash
|
|
PORT=3000 ./sitemap-api
|
|
```
|
|
|
|
### Build errors
|
|
```bash
|
|
go mod tidy
|
|
go clean -cache
|
|
go build -o sitemap-api .
|
|
```
|
|
|
|
### Database locked
|
|
```bash
|
|
rm sitemap.db
|
|
./sitemap-api
|
|
```
|
|
|
|
### CGO errors
|
|
Make sure you have gcc installed:
|
|
- **Ubuntu/Debian**: `sudo apt-get install build-essential`
|
|
- **macOS**: `xcode-select --install`
|
|
- **Windows**: Install MinGW or TDM-GCC
|
|
|
|
## Next Steps
|
|
|
|
- Read the full [README.md](README.md) for details
|
|
- Customize the crawler in `crawler/crawler.go`
|
|
- Add authentication to handlers
|
|
- Deploy to production (see README for nginx config)
|
|
- Add more metadata tracking
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
sitemap-api/
|
|
├── main.go # Server entry point
|
|
├── handlers/ # HTTP handlers & SSE
|
|
├── crawler/ # Web crawler logic
|
|
├── database/ # SQLite operations
|
|
├── models/ # Data structures
|
|
├── static/ # Frontend (served at /)
|
|
├── README.md # Full documentation
|
|
├── run.sh # Quick start script
|
|
├── Makefile # Build commands
|
|
└── Dockerfile # Container setup
|
|
```
|
|
|
|
## Support
|
|
|
|
Having issues? Check:
|
|
1. Go version >= 1.21
|
|
2. Port 8080 is available
|
|
3. SQLite3 is working
|
|
4. All dependencies installed
|
|
|
|
Still stuck? Open an issue on GitHub!
|
|
|
|
---
|
|
|
|
**Built with ❤️ using Go + Goroutines + Server-Sent Events**
|