Spitfire/Search
Spitfire/Search
1
0
Fork 0
Code Issues 7 Pull requests Projects Releases Packages Wiki Activity Actions

Updated Config.md with new variables

partisan 2025-01-12 16:47:34 +01:00
parent a5de655db8
commit 3ec5e79958
1 changed files with 59 additions and 38 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

97
Config.md

@ -3,54 +3,75 @@
This page documents all configuration variables available in `config.ini`, including their purpose, possible values, and default settings.
## Server Configuration
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|--------------|-------------------------------------------------|-------------------|---------------------------|
| `Port` | Port number for the application server | `5000` | Any valid port number |
| `Domain` | Domain address of the server | `localhost` | Any valid domain string |
| `LogLevel` | Logging level for the application | `1` | `0` (Silent), `1` (Errors), `2` (Warnings), `3` (Info), `4` (Debug) |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| -------------- | ---------------------------------------- | ------------------- | --------------------------------------------------------------------- |
| `Port` | Port number for the application server | `5000` | Any valid port number |
| `Domain` | Domain address of the server | `localhost` | Any valid domain name or IP address |
| `LogLevel` | Logging level for the application | `1` | `0` (Silent), `1` (Errors), `2` (Warnings), `3` (Info), `4` (Debug) |
## Peer Configuration
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|--------------|-------------------------------------------------|-------------------|---------------------------|
| `AuthCode` | Authorization code for peer connections | Auto-generated | 64-character random string |
| `PeerID` | Unique identifier for the peer | Auto-generated | Any unique string |
| `Peers` | Comma-separated list of peer addresses | Empty | List of peer URLs |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| -------------- | ----------------------------------------- | ------------------- | ---------------------------- |
| `AuthCode` | Authorization code for peer connections | Auto-generated | 64-character random string |
| `PeerID` | Unique identifier for the peer | Auto-generated | Any unique string |
| `Peers` | Comma-separated list of peer addresses | Empty | List of peer URLs |
## Feature Toggles
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|-----------------------|---------------------------------------------------------|-------------------|---------------------------|
| `NodesEnabled` | Enable or disable peer-to-peer nodes | `false` | `true` / `false` |
| `MetaSearchEnabled` | Enable metasearch crawling from other search engines | `true` | `true` / `false` |
| `IndexerEnabled` | Enable web crawling and indexing | `false` | `true` / `false` |
| `WebsiteEnabled` | Enable website hosting feature | `true` | `true` / `false` |
| `RamCacheEnabled` | Enable caching in RAM | `true` | `true` / `false` |
| `DriveCacheEnabled` | Enable caching on the disk | `false` | `true` / `false` |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| ---------------- | ------------------------------------------------------ | ------------------- | --------------------- |
| `Nodes` | Enable or disable peer-to-peer nodes | `false` | `true` / `false` |
| `MetaSearch` | Enable metasearch crawling from other search engines | `true` | `true` / `false` |
| `Indexer` | Enable web crawling and indexing | `false` | `true` / `false` |
| `Website` | Enable website hosting feature | `true` | `true` / `false` |
| `RamCache` | Enable caching in RAM | `true` | `true` / `false` |
| `DriveCache` | Enable caching on the disk | `false` | `true` / `false` |
| `MetaProxy` | Use proxies for metasearch crawling | `false` | `true` / `false` |
| `CrawlerProxy` | Use proxies for general web crawling and indexing | `false` | `true` / `false` |
## Proxy Configuration
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|-----------------------|---------------------------------------------------------|-------------------|---------------------------|
| `MetaProxyEnabled` | Use proxies for metasearch crawling | `false` | `true` / `false` |
| `CrawlerProxyEnabled` | Use proxies for general web crawling and indexing | `false` | `true` / `false` |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| ---------------------- | ------------------------------------------------------ | ------------------- | ------------------------- |
| `MetaProxyStrict` | Enforce strict behavior when MetaProxy fails | `true` | `true` / `false` |
| `MetaProxies` | List of proxies to use for metasearch crawling | Empty | Comma-separated proxies |
| `CrawlerProxyStrict` | Enforce strict behavior when CrawlerProxy fails | `true` | `true` / `false` |
| `CrawlerProxies` | List of proxies to use for web crawling and indexing | Empty | Comma-separated proxies |
### Notes:
- Proxies must be specified as a **comma-separated list**.
- Each proxy should be in the format:
- `ADDRESS:PORT`
- `ADDRESS:PORT:USER:PASSWORD`
- Example:
- `proxy1.example.com:8080:user1:pass1, proxy2.example.com:8080`
## Crawling and Indexing
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|----------------------------|-----------------------------------------------------|-------------------|---------------------------|
| `ConcurrentStandardCrawlers` | Number of concurrent standard crawlers | `12` | Positive integer |
| `ConcurrentChromeCrawlers` | Number of concurrent Chrome-based crawlers | `4` | Positive integer |
| `CrawlingInterval` | Interval for refreshing crawled results | `24h` | Any valid Go duration |
| `MaxPagesPerDomain` | Maximum number of pages to crawl per domain | `10` | Positive integer |
| `IndexBatchSize` | Number of pages to index in a single batch | `50` | Positive integer |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| ------------------------------ | --------------------------------------------- | ------------------- | ----------------------------------- |
| `ConcurrentStandardCrawlers` | Number of standard crawlers running at once | `12` | Positive number |
| `ConcurrentChromeCrawlers` | Number of Chrome-based crawlers running | `4` | Positive number |
| `CrawlingInterval` | Time between refreshing crawled results | `24h` | Duration like`1h`, `30m`, `2h45m` |
| `MaxPagesPerDomain` | Maximum pages to crawl per domain | `10` | Positive number |
| `IndexBatchSize` | Number of pages to index in one batch | `50` | Positive number |
## Cache Configuration
### RAM Cache
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|----------------------|-------------------------------------|-------------------|---------------------------|
| `Duration` | Duration to store results in RAM | `6h` | Any valid Go duration |
| `MaxUsage` | Maximum usage of RAM cache | `90%` | Percentage (`%`) or size (e.g., `2GiB`) |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| -------------- | ------------------------------ | ------------------- | ----------------------------------------- |
| `Duration` | Time to store results in RAM | `6h` | Duration like`1h`, `30m`, `2h45m` |
| `MaxUsage` | Maximum RAM usage | `90%` | Percentage (`%`) or size (e.g., `2GiB`) |
### Drive Cache
| **Variable** | **Description** | **Default Value** | **Possible Values** |
|----------------------|-------------------------------------|-------------------|---------------------------|
| `Duration` | Duration to store results on disk | `48h` | Any valid Go duration |
| `MaxUsage` | Maximum usage of drive cache | `90%` | Percentage (`%`) or size (e.g., `10GiB`) |
| `Path` | Path to the drive cache directory | `./cache` | Any valid file path |
| **Variable** | **Description** | **Default Value** | **Possible Values** |
| -------------- | ---------------------------------- | ------------------- | ------------------------------------------ |
| `Duration` | Time to store results on disk | `48h` | Duration like`1h`, `30m`, `2h45m` |
| `MaxUsage` | Maximum disk usage | `90%` | Percentage (`%`) or size (e.g., `10GiB`) |
| `Path` | Path to the disk cache directory | `./cache` | Any valid file path |