Skip to content

Commit

Permalink
Update go.mod github.com/gocolly/colly/v2
Browse files Browse the repository at this point in the history
  • Loading branch information
sky126 committed Dec 10, 2019
1 parent b5c8aae commit b83ae6d
Show file tree
Hide file tree
Showing 36 changed files with 48 additions and 47 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ We welcome any type of contribution, not only code. You can help with
- **QA**: file bug reports, the more details you can give the better (e.g. screenshots with the console open)
- **Marketing**: writing blog posts, howto's, printing stickers, ...
- **Community**: presenting the project at meetups, organizing a dedicated meetup for the local community, ...
- **Code**: take a look at the [open issues](https://github.com/gocolly/colly/issues). Even if you can't write code, commenting on them, showing that you care about a given issue matters. It helps us triage them.
- **Code**: take a look at the [open issues](https://github.com/gocolly/colly/v2/issues). Even if you can't write code, commenting on them, showing that you care about a given issue matters. It helps us triage them.
- **Money**: we welcome financial contributions in full transparency on our [open collective](https://opencollective.com/colly).

## Your First Contribution
Expand All @@ -31,7 +31,7 @@ Anyone can file an expense. If the expense makes sense for the development of th

## Questions

If you have any questions, create an [issue](https://github.com/gocolly/colly/issues/new) (protip: do a quick search first to see if someone else didn't ask the same question before!).
If you have any questions, create an [issue](https://github.com/gocolly/colly/v2/issues/new) (protip: do a quick search first to see if someone else didn't ask the same question before!).
You can also reach us at [email protected].

## Credits
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ Colly provides a clean interface to write any kind of crawler/scraper/spider.

With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.

[![GoDoc](https://godoc.org/github.com/gocolly/colly?status.svg)](https://godoc.org/github.com/gocolly/colly)
[![GoDoc](https://godoc.org/github.com/gocolly/colly/v2?status.svg)](https://godoc.org/github.com/gocolly/colly/v2)
[![Backers on Open Collective](https://opencollective.com/colly/backers/badge.svg)](#backers) [![Sponsors on Open Collective](https://opencollective.com/colly/sponsors/badge.svg)](#sponsors) [![build status](https://img.shields.io/travis/gocolly/colly/master.svg?style=flat-square)](https://travis-ci.org/gocolly/colly)
[![report card](https://img.shields.io/badge/report%20card-a%2B-ff3333.svg?style=flat-square)](http://goreportcard.com/report/gocolly/colly)
[![view examples](https://img.shields.io/badge/learn%20by-examples-0077b3.svg?style=flat-square)](https://github.com/gocolly/colly/tree/master/_examples)
[![view examples](https://img.shields.io/badge/learn%20by-examples-0077b3.svg?style=flat-square)](https://github.com/gocolly/colly/v2/tree/master/_examples)
[![Code Coverage](https://img.shields.io/codecov/c/github/gocolly/colly/master.svg)](https://codecov.io/github/gocolly/colly?branch=master)
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fgocolly%2Fcolly.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Fgocolly%2Fcolly?ref=badge_shield)
[![Twitter URL](https://img.shields.io/badge/twitter-follow-green.svg)](https://twitter.com/gocolly)
Expand Down Expand Up @@ -49,19 +49,19 @@ func main() {
}
```

See [examples folder](https://github.com/gocolly/colly/tree/master/_examples) for more detailed examples.
See [examples folder](https://github.com/gocolly/colly/v2/tree/master/_examples) for more detailed examples.


## Installation

```
go get -u github.com/gocolly/colly/...
go get -u github.com/gocolly/colly/v2/...
```


## Bugs

Bugs or suggestions? Visit the [issue tracker](https://github.com/gocolly/colly/issues) or join `#colly` on freenode
Bugs or suggestions? Visit the [issue tracker](https://github.com/gocolly/colly/v2/issues) or join `#colly` on freenode


## Other Projects Using Colly
Expand All @@ -84,7 +84,7 @@ If you are using Colly in a project please send a pull request to add it to the
## Contributors

This project exists thanks to all the people who contribute. [[Contribute]](CONTRIBUTING.md).
<a href="https://github.com/gocolly/colly/graphs/contributors"><img src="https://opencollective.com/colly/contributors.svg?width=890" /></a>
<a href="https://github.com/gocolly/colly/v2/graphs/contributors"><img src="https://opencollective.com/colly/contributors.svg?width=890" /></a>


## Backers
Expand Down
2 changes: 1 addition & 1 deletion _examples/basic/basic.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/coursera_courses/coursera_courses.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"os"
"strings"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

// Course stores information about a coursera course
Expand Down
2 changes: 1 addition & 1 deletion _examples/cryptocoinmarketcap/cryptocoinmarketcap.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"log"
"os"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/error_handling/error_handling.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/factba.se/factbase.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"io/ioutil"
"strconv"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

var baseSearchURL = "https://factba.se/json/json-transcript.php?q=&f=&dt=&p="
Expand Down
2 changes: 1 addition & 1 deletion _examples/google_groups/google_groups.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"os"
"strings"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

// Mail is the container of a single e-mail
Expand Down
2 changes: 1 addition & 1 deletion _examples/hackernews_comments/hackernews_comments.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"strconv"
"strings"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

type comment struct {
Expand Down
2 changes: 1 addition & 1 deletion _examples/instagram/instagram.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
"regexp"
"strings"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

// "id": user id, "after": end cursor
Expand Down
2 changes: 1 addition & 1 deletion _examples/local_files/local_files.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"os"
"path/filepath"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/login/login.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"log"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/max_depth/max_depth.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/multipart/multipart.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (
"os"
"time"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func generateFormData() map[string][]byte {
Expand Down
2 changes: 1 addition & 1 deletion _examples/openedx_courses/openedx_courses.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"strings"
"time"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

// DATE_FORMAT default format date used in openedx
Expand Down
2 changes: 1 addition & 1 deletion _examples/parallel/parallel.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
4 changes: 2 additions & 2 deletions _examples/proxy_switcher/proxy_switcher.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ import (
"bytes"
"log"

"github.com/gocolly/colly"
"github.com/gocolly/colly/proxy"
"github.com/gocolly/colly/v2"
"github.com/gocolly/colly/v2/proxy"
)

func main() {
Expand Down
4 changes: 2 additions & 2 deletions _examples/queue/queue.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/queue"
"github.com/gocolly/colly/v2"
"github.com/gocolly/colly/v2/queue"
)

func main() {
Expand Down
4 changes: 2 additions & 2 deletions _examples/random_delay/random_delay.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ import (
"fmt"
"time"

"github.com/gocolly/colly"
"github.com/gocolly/colly/debug"
"github.com/gocolly/colly/v2"
"github.com/gocolly/colly/v2/debug"
)

func main() {
Expand Down
4 changes: 2 additions & 2 deletions _examples/rate_limit/rate_limit.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/debug"
"github.com/gocolly/colly/v2"
"github.com/gocolly/colly/v2/debug"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/reddit/reddit.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"os"
"time"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

type item struct {
Expand Down
2 changes: 1 addition & 1 deletion _examples/request_context/request_context.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/scraper_server/scraper_server.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"log"
"net/http"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

type pageInfo struct {
Expand Down
2 changes: 1 addition & 1 deletion _examples/shopify_sitemap/shopify_sitemap.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package main
import (
"fmt"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/url_filter/url_filter.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"fmt"
"regexp"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion _examples/xkcd_store/xkcd_store.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"log"
"os"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

func main() {
Expand Down
2 changes: 1 addition & 1 deletion cmd/colly/colly.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ var scraperHeadTemplate = `package main
import (
"log"
"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)
func main() {
Expand Down
6 changes: 3 additions & 3 deletions colly.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ import (
"github.com/PuerkitoBio/goquery"
"github.com/antchfx/htmlquery"
"github.com/antchfx/xmlquery"
"github.com/gocolly/colly/debug"
"github.com/gocolly/colly/storage"
"github.com/gocolly/colly/v2/debug"
"github.com/gocolly/colly/v2/storage"
"github.com/kennygrant/sanitize"
"github.com/temoto/robotstxt"
"google.golang.org/appengine/urlfetch"
Expand Down Expand Up @@ -368,7 +368,7 @@ func Debugger(d debug.Debugger) CollectorOption {
// Init initializes the Collector's private variables and sets default
// configuration for the Collector
func (c *Collector) Init() {
c.UserAgent = "colly - https://github.com/gocolly/colly"
c.UserAgent = "colly - https://github.com/gocolly/colly/v2"
c.MaxDepth = 0
c.store = &storage.InMemoryStorage{}
c.store.Init()
Expand Down
2 changes: 1 addition & 1 deletion colly_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import (

"github.com/PuerkitoBio/goquery"

"github.com/gocolly/colly/debug"
"github.com/gocolly/colly/v2/debug"
)

var serverIndexResponse = []byte("hello world\n")
Expand Down
2 changes: 1 addition & 1 deletion extensions/random_user_agent.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"fmt"
"math/rand"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

var uaGens = []func() string{
Expand Down
2 changes: 1 addition & 1 deletion extensions/referer.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package extensions

import (
"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

// Referer sets valid Referer HTTP header to requests.
Expand Down
2 changes: 1 addition & 1 deletion extensions/url_length_filter.go
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
package extensions

import (
"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

// URLLengthFilter filters out requests with URLs longer than URLLengthLimit
Expand Down
1 change: 1 addition & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=
github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
github.com/gocolly/colly v1.2.0 h1:qRz9YAn8FIH0qzgNUw+HT9UN7wm1oF9OBAilwEWpyrI=
github.com/golang/protobuf v1.3.1 h1:YF8+flBXS5eO826T4nzqPrxfhQThhXl0YzfuUPu4SBg=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/jawher/mow.cli v1.1.0 h1:NdtHXRc0CwZQ507wMvQ/IS+Q3W3x2fycn973/b8Zuk8=
Expand Down
2 changes: 1 addition & 1 deletion proxy/proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"net/url"
"sync/atomic"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

type roundRobinSwitcher struct {
Expand Down
2 changes: 1 addition & 1 deletion queue/queue.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
"sync"
"sync/atomic"

"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
)

const stop = true
Expand Down
2 changes: 1 addition & 1 deletion xmlelement_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ package colly_test

import (
"github.com/antchfx/htmlquery"
"github.com/gocolly/colly"
"github.com/gocolly/colly/v2"
"reflect"
"strings"
"testing"
Expand Down

0 comments on commit b83ae6d

Please sign in to comment.