View on GitHub

Validate-website

Web crawler for checking the validity of your documents.

Download this project as a .zip file Download this project as a tar.gz file

validate-website

Description

Web crawler for checking the validity of your documents

validate website

Installation

Debian

apt install ruby-dev libxslt-dev libxml2-dev

RubyGems

gem install validate-website

Synopsis

validate-website [OPTIONS]
validate-website-static [OPTIONS]

Description

validate-website is a web crawler for checking the markup validity with XML Schema / DTD and not found urls (more info doc/validate-website.adoc).

validate-website-static checks the markup validity of your local documents with XML Schema / DTD (more info doc/validate-website-static.adoc).

HTML5 support with Validator.nu Web Service.

Exit status

0: Markup is valid and no 404 found.
64: Not valid markup found.
65: There are pages not found.
66: There are not valid markup and pages not found.

On your application

require 'validate_website/validator'
body = '<!DOCTYPE html><html></html>'
v = ValidateWebsite::Validator.new(Nokogiri::HTML(body), body)
v.valid? # => false

Jekyll static site validation

You can add this Rake task to validate a jekyll site:

desc 'validate _site with validate website'
task validate: :build do
  Dir.chdir('_site') do
    sh("validate-website-static --site '<CONFIG_URL>'") do |ok, res|
      unless ok
        puts "validate error (status = #{res.exitstatus})"
      end
    end
  end
end

Tests

With standard environment:

bundle exec rake

Credits

Thanks tenderlove for Nokogiri, this tool is inspired from markup_validity.
And Chris Kite for Anemone web-spider framework and postmodern for Spidr.

More info

HTML5 validator web service

The HTML5 support is done by using the Validator.nu Web Service, so the content of your webpage is logged by a tier. It's not the case for other validation because validate-website use the XML Schema or DTD stored on the data/ directory.

Please read http://about.validator.nu/#tos for more info on the HTML5 validation service.

Use validator standalone web server locally

You can download validator jar and start it with:

java -cp PATH_TO/vnu.jar nu.validator.servlet.Main 8888

Then you can use validate-website option:

--html5-validator-service-url http://localhost:8888/
# or
export VALIDATOR_NU_URL="http://localhost:8888/"

This will prevent you to be blacklisted from validator webservice.

Contributors

See GitHub.

License

The MIT License