package scanner import ( "fmt" "io/ioutil" "net/http" "regexp" "strings" ) // Public definition of a Scanner type Scanner struct { Client *http.Client } // Internal interface for Scanner type scanner interface { Scan(url string, regExp regexp.Regexp) (bool, error) } // Given a URL and regular expression return if it matches along with any error // NOTE: In case of an error this will return false func (s *Scanner) Scan(url string, regExpString string) (bool, error) { regExp, err := regexp.Compile(regExpString) if err != nil { return false, err } resp, err := s.Client.Get(url) if err != nil { if strings.Contains(err.Error(), "device or resource busy") { fmt.Println("If you see this stop execution and recompile with netgo") return false, nil } if strings.Contains(err.Error(), "too many open files") { fmt.Println("If you see this stop execution then increase 'ulimit -n'") return false, nil } return false, err } defer resp.Body.Close() // Ensure body is closed at the end of the function body, err := ioutil.ReadAll(resp.Body) if err != nil { return false, err } return regExp.Match(body), nil }