kar/sitemap-generator-xml-golang

Fork 0

Files

Kar@k5 10b19d4ed6 init

2026-02-05 19:13:45 +05:30

3.2 KiB

Raw Blame History

🚀 Quick Start Guide

Get your sitemap generator running in 3 steps!

Step 1: Install Go

If you don't have Go installed:

Download from https://golang.org/dl/
Install Go 1.21 or later
Verify: go version

Step 2: Run the Application

Option A: Using the run script (easiest)

cd sitemap-api
./run.sh

Option B: Using Make

cd sitemap-api
make run

Option C: Manual

cd sitemap-api
go mod download
go build -o sitemap-api .
./sitemap-api

Step 3: Use the Application

Open your browser → http://localhost:8080
Enter a URL → e.g., https://example.com
Set crawl depth → 1-5 (default: 3)
Click "Generate Sitemap" → Watch real-time progress!
Download XML → Click the download button when complete

Testing Multiple Users

Open multiple browser tabs to http://localhost:8080 and start different crawls simultaneously. Each will have its own UUID and progress stream!

API Usage Examples

Start a crawl

curl -X POST http://localhost:8080/generate-sitemap-xml \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "max_depth": 3}'

Response:

{
  "uuid": "550e8400-e29b-41d4-a716-446655440000",
  "site_id": 123,
  "status": "processing",
  "stream_url": "/stream/550e8400-e29b-41d4-a716-446655440000",
  "message": "Sitemap generation started"
}

Monitor progress (SSE)

curl http://localhost:8080/stream/550e8400-e29b-41d4-a716-446655440000

Download sitemap

curl http://localhost:8080/download/550e8400-e29b-41d4-a716-446655440000 -o sitemap.xml

List all sitemaps

curl http://localhost:8080/sites

Delete a sitemap

curl -X DELETE http://localhost:8080/sites/123

Troubleshooting

Port already in use

PORT=3000 ./sitemap-api

Build errors

go mod tidy
go clean -cache
go build -o sitemap-api .

Database locked

rm sitemap.db
./sitemap-api

CGO errors

Make sure you have gcc installed:

Ubuntu/Debian: sudo apt-get install build-essential
macOS: xcode-select --install
Windows: Install MinGW or TDM-GCC

Next Steps

Read the full README.md for details
Customize the crawler in crawler/crawler.go
Add authentication to handlers
Deploy to production (see README for nginx config)
Add more metadata tracking

Project Structure

sitemap-api/
├── main.go              # Server entry point
├── handlers/            # HTTP handlers & SSE
├── crawler/             # Web crawler logic
├── database/            # SQLite operations
├── models/              # Data structures
├── static/              # Frontend (served at /)
├── README.md           # Full documentation
├── run.sh              # Quick start script
├── Makefile            # Build commands
└── Dockerfile          # Container setup

Support

Having issues? Check:

Go version >= 1.21
Port 8080 is available
SQLite3 is working
All dependencies installed

Still stuck? Open an issue on GitHub!

Built with ❤️ using Go + Goroutines + Server-Sent Events

3.2 KiB Raw Blame History