hq-go-url

hq-go-url is a Go (Golang) package for extracting, parsing, and manipulating URLs with ease. It provides powerful tools to:

• Extract URLs from text using flexible, customizable regular expressions.
• Parse URLs with extended functionality that includes detailed domain analysis—splitting a domain into its subdomain, second-level domain (SLD), and top-level domain (TLD).

This package is especially useful for developers who need to work with URLs in a structured way.

Resources

• Features
• Usage
  • Extraction
  • Parsing
    • Domains
    • URLs
• Contributing
• Licensing

Features

• Configurable URL Extraction: Extract URLs from text using regular expressions.
• Domain Parsing: Parse domains into subdomains, second-level domains, and top-level domains.
• Extended URL Parsing: Extend the standard net/url package in Go with additional fields and capabilities.

Usage

go get -v -u go.source.hueristiq.com/url

Below are examples demonstrating how to use the different features of the hq-go-url package.

Extraction

The extractor package allows you to extract URLs from a block of text using regular expressions that can be customized to your needs.

package main

import (
    "fmt"
    "log"

    "go.source.hueristiq.com/url/extractor"
)

func main() {
    // Create a new extractor with the requirement that URLs include a scheme.
    ex := extractor.New(extractor.WithScheme())

    // Compile the regular expression used for URL extraction.
    regex := ex.CompileRegex()

    text := "Visit our website at https://www.example.com or contact us at [email protected]."

    // Extract all matching URLs from the text.
    urls := regex.FindAllString(text, -1)

    fmt.Println("Extracted URLs:")

    for _, u := range urls {

        fmt.Println(u)
    }
}

You can customize how URLs are extracted by specifying URL schemes, hosts, or providing custom regular expression patterns.

• Extract URLs with Schemes Pattern:

   e := extractor.New(
   	extractor.WithSchemePattern(`(?:https?|ftp)://`),
   )

  This configuration will extract URLs with http, https, or ftp schemes.

• Extract URLs with Host Pattern:

   e := extractor.New(
   	extractor.WithHostPattern(`(?:www\.)?example\.com`),
   )

  This configuration will extract URLs that have hosts matching www.example.com or example.com.

Parsing

Domains

package main

import (
	"fmt"

	"go.source.hueristiq.com/url/parser"
)

func main() {
	p := parser.NewDomainParser()

	parsed := p.Parse("subdomain.example.com")

	fmt.Printf("Subdomain: %s, SLD: %s, TLD: %s\n", parsed.Subdomain, parsed.SLD, parsed.TLD)
}

URLs

package main

import (
	"fmt"

	"go.source.hueristiq.com/url/parser"
)

func main() {
	p := parser.NewURLParser()

	parsed, err := p.Parse("https://subdomain.example.com:8080/path/file.txt")
	if err != nil {
		fmt.Println("Error parsing URL:", err)

		return
	}

	fmt.Printf("Scheme: %s\n", parsed.Scheme)
	fmt.Printf("Host: %s\n", parsed.Host)
	fmt.Printf("Hostname: %s\n", parsed.Hostname())
	fmt.Printf("Subdomain: %s\n", parsed.Domain.Subdomain)
	fmt.Printf("SLD: %s\n", parsed.Domain.SLD)
	fmt.Printf("TLD: %s\n", parsed.Domain.TLD)
	fmt.Printf("Port: %s\n", parsed.Port())
	fmt.Printf("Path: %s\n", parsed.Path)
}

Set a default scheme:

p := parser.NewURLParser(parser.URLParserWithDefaultScheme("https"))

Contributing

Feel free to submit Pull Requests or report Issues. For more details, check out the contribution guidelines.

Huge thanks to the contributors thus far!

Licensing

This package is licensed under the MIT license. You are free to use, modify, and distribute it, as long as you follow the terms of the license. You can find the full license text in the repository - Full MIT license text.