Grawlr

Grawlr, a simple web crawler written in Go

Table of Contents

• Installation
• Usage
• Testing
• Linting

Installation

Prerequisites

• Go (version 1.23+)

Clone the Repository

To download the source code, clone the repository:

git clone git@github.com:HRemonen/Grawlr.git
cd grawlr

Install Dependencies

Run the following command to install Go module dependencies:

go mod tidy

This will install any necessary packages for the project.

Usage

For detailed usage instructions, refer to the Usage Documentation.

Testing

This project includes tests for various different modules.

To run the tests for a single package, use the command:

go test -v <package name dir>

To run all the tests:

go test ./...

Linting

To ensure that the codebase follows Go best practices and maintain a clean, consistent style, we use golangci-lint, a popular linter aggregator for Go.

Installing the Linter

First, install golangci-lint by following the official instructions here. You can also install it using go install:

go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest

Running the Linter

Once installed, you can run the linter on the project using the following command:

golangci-lint run

This will check the entire codebase for issues and display any linting errors, warnings, or suggestions.

In some cases, the linter can automatically fix issues like formatting errors. To apply fixes automatically, run:

golangci-lint run --fix

The linter is also run on the CI pipeline.