Introduction
Description
Codename SCNR is a modular, distributed, high-performance DAST web application security scanner framework, capable of analyzing the behavior and security of modern web applications and web APIs.
You can access Codename SCNR via multiple interfaces, such as:
Back-end support
A wide range of back-end technologies is supported, including:
- Operating systems
- BSD
- Linux
- Unix
- Windows
- Solaris
- Databases
- SQL
- MySQL
- PostreSQL
- MSSQL
- Oracle
- SQLite
- Ingres
- EMC
- DB2
- Interbase
- Informix
- Firebird
- MaxDB
- Sybase
- Frontbase
- HSQLDB
- Access
- NoSQL
- MongoDB
- SQL
- Web servers
- Apache
- IIS
- Nginx
- Tomcat
- Jetty
- Gunicorn
- Programming languages
- PHP
- ASP
- ASPX
- Java
- Python
- Ruby
- Javascript
- Frameworks
- Rack
- CakePHP
- Rails
- Django
- ASP.NET MVC
- JSF
- CherryPy
- Nette
- Symfony
- NodeJS
- Express
This list keeps growing but new platforms or failure to fingerprint supported ones don’t disable the Codename SCNR engine, they merely force it to be more extensive in its scan.
Upon successful identification or configuration of platform types, the scan will be much more focused, less resource intensive and require less time to complete.
Front-end support
HTML5, modern Javascript APIs and modern DOM APIs are supported by basing their execution and analysis on Google Chromium.
Codename SCNR injects a custom environment to monitor JS objects and APIs in order to trace execution and data flows and thus provide highly in-depth reporting as to how a client-side security issue was identified which also greatly assists in its remediation.
Behavioral analysis
Codename SCNR will study the web application/service to identify how each input interacts with the front and back ends and tailor the audit for each specific input’s characteristics.
This results in highly self-optimized scans using less resources and requiring less time to complete, as well as less server stress.
Training also continues during the audit process and new inputs that may appear during that time will be incorporated into the scan in whole.
Extendability
Its modular architecture allows for easy augmentation when it comes to security checks, arbitrary custom functionality in the form of plugins and bespoke reporting.
Entities which perform tasks crucial to the operation of a web scanner have been abstracted to be components, more to be easily added by anyone in order to extend functionality.
Components are split into the following types:
- Checks – Security checks.
- Active – They actively engage the web application via its inputs.
- Passive – They passively look for objects.
- Plugins – Add arbitrary functionality to the system, accept options and run in parallel to the scan.
- Reporters – They export the scan results in several formats.
- Path extractors – They extract paths for the crawler to follow.
- Fingerprinters – They identify OS version, platforms, servers, etc.
Customization
Furthermore, scripted scans allow for the creation of basically tailor made scans by moving decision making points and configuration to user-specified methods and can extend to even creating a custom scanner for any web application backed by the Codename SCNR engine.
The API is tidy and simple and easily allows you to plug-in to key API1 scan points in order to get the best results from any scan.
Scripts are written in Ruby and can thus be stored in your favorite CVS, this enables you to work side-by-side with the web application development team and have the right script revision alongside the respective web application revision.
Scalability
No dependencies, no configuration; Codename SCNR can build a cloud of itself that allows you to scale both horizontally and vertically.
Scale up by plugging more nodes to its Grid, or down by unplugging them.
Furthermore, with multi-Instance scans you can not only distribute multiple scans across nodes, but also individual scans, for super fast scanning of large sites.
Finally, with its quick suspend-to-disk/restore feature, running scans can easily be moved from node to node, accommodating highly optimized load-balancing and cost saving policies.
Deployment
Deployment options range from command-line utilities for direct scans, scripted scans (for configuration and custom scanners) as well as distributed deployments to perform scans from remote hosts and Grid/cloud/SaaS setups.
Its simple distributed architecture2 allows for easy creation of self-healing, load-balanced (vertically and horizontally) scanner grids; basically allowing for the creation of private scanner clouds in either yours or a Cloud provider’s infrastructure.
Conclusion
Thus, Codename SCNR can in essence fit into any SDLC with great grace, ease and little care.
API/script functionality is provided by DSeL.
Distributed functionality is provided by Cuboid.
Installation
For installation instructions please refer to the installer.
System requirements
| Operating System | Architecture | RAM | Disk | CPU |
|---|---|---|---|---|
| Linux | x86 64bit | 2GB | 4GB | Multicore |
Resource constrained environments
To optimize the resources a scan may use please consult:
In addition, Agents and other servers can have their max-slots adjusted
to a user-specified value, instead of the default, which is auto and based
on the aforementioned system requirements.
Please issue the -h flag to see available options for each executable in order
to examine the applicable overrides.
Direct
The easiest approach is a direct scan using the scnr CLI executable.
To see all available options run:
bin/scnr -h
Example
The following command will run a scan with default settings against http://testhmtml5.vulnweb.com.
bin/scnr http://testhmtml5.vulnweb.com
Scripted
Scripted scans allow you to configure the system and take over decision making points for a much more fine-grained scan. Aside from that, scripts also allow you to quickly add custom components on the fly.
Scan scripts can either be a form of configuration or standalone scanners.
Examples
As configuration
With helpers
html5.config.rb:
SCNR::Application::API.run do
require '/home/user/script/helpers'
Dom {
on :event, &method(:on_event_handler)
}
Checks {
# This will run from the context of SCNR::Engine::Check::Base; it
# basically creates a new check component on the fly.
as :not_found, check_404_info, method(:check_404)
}
Plugins {
# This will run from the context of SCNR::Engine::Plugin::Base; it
# basically creates a new plugin component on the fly.
as :my_plugin, my_plugin_info, method(:my_plugin)
}
Scan {
Session {
to :login, &method(:login)
to :check, &method(:login_check)
}
Scope {
# Don't visit resources that will end the session.
reject :url, &method(:to_logout)
}
}
end
helpers.rb:
# Allow some time for the modal animation to complete in order for
# the login form to appear.
#
# (Not actually necessary, this is just an example on how to hande quirks.)
def on_event_handler( result, locator, event, options, browser )
return if locator.attributes['href'] != '#myModal' || event != :click
sleep 1
end
# Does something really simple, logs an issue for each 404 page.
def check_404
response = page.response
return if response.code != 404
log(
proof: response.status_line,
vector: SCNR::Engine::Element::Server.new( response.url ),
response: response
)
end
def check_404_info
{
issue: {
name: 'Page not found',
severity: SCNR::Engine::Issue::Severity::INFORMATIONAL
}
}
end
def my_plugin
# Do stuff then wait until scan completes.
wait_while_framework_running
# Do stuff after scan completes.
end
def my_plugin_info
{
name: 'My Plugin',
description: 'Just waits for the scan to finish,'
}
end
def login( browser )
# Login with whichever interface you prefer.
watir = browser.watir
selenium = browser.selenium
watir.goto SCNR::Engine::Options.url
watir.link( href: '#myModal' ).click
form = watir.form( id: 'loginForm' )
form.text_field( name: 'username' ).set 'admin'
form.text_field( name: 'password' ).set 'admin'
form.submit
end
def login_check( &in_async_mode )
http_client = SCNR::Engine::HTTP::Client
check = proc { |r| r.body.optimized_include? '<b>admin' }
# If an async block is passed, then the framework would rather
# schedule it to run asynchronously.
if in_async_mode
http_client.get SCNR::Engine::Options.url do |response|
in_async_mode.call check.call( response )
end
else
response = http_client.get( SCNR::Engine::Options.url, mode: :sync )
check.call( response )
end
end
def to_logout( url )
url.path.optimized_include?( 'login' ) ||
url.path.optimized_include?( 'logout' )
end
Single file
SCNR::Application::API.run do
Dom {
# Allow some time for the modal animation to complete in order for
# the login form to appear.
#
# (Not actually necessary, this is just an example on how to hande quirks.)
on :event do |_, locator, event, *|
next if locator.attributes['href'] != '#myModal' || event != :click
sleep 1
end
}
Checks {
# This will run from the context of SCNR::Engine::Check::Base; it
# basically creates a new check component on the fly.
#
# Does something really simple, logs an issue for each 404 page.
as :not_found,
issue: {
name: 'Page not found',
severity: SCNR::Engine::Issue::Severity::INFORMATIONAL
} do
response = page.response
next if response.code != 404
log(
proof: response.status_line,
vector: SCNR::Engine::Element::Server.new( response.url ),
response: response
)
end
}
Plugins {
# This will run from the context of SCNR::Engine::Plugin::Base; it
# basically creates a new plugin component on the fly.
as :my_plugin do
# Do stuff then wait until scan completes.
wait_while_framework_running
# Do stuff after scan completes.
end
}
Scan {
Session {
to :login do |browser|
# Login with whichever interface you prefer.
watir = browser.watir
selenium = browser.selenium
watir.goto SCNR::Engine::Options.url
watir.link( href: '#myModal' ).click
form = watir.form( id: 'loginForm' )
form.text_field( name: 'username' ).set 'admin'
form.text_field( name: 'password' ).set 'admin'
form.submit
end
to :check do |async|
http_client = SCNR::Engine::HTTP::Client
check = proc { |r| r.body.optimized_include? '<b>admin' }
# If an async block is passed, then the framework would rather
# schedule it to run asynchronously.
if async
http_client.get SCNR::Engine::Options.url do |response|
success = check.call( response )
async.call success
end
else
response = http_client.get( SCNR::Engine::Options.url, mode: :sync )
check.call( response )
end
end
}
Scope {
# Don't visit resources that will end the session.
reject :url do |url|
url.path.optimized_include?( 'login' ) ||
url.path.optimized_include?( 'logout' )
end
}
}
end
Supposing the above is saved as html5.config.rb:
bin/scnr http://testhtml5.vulnweb.com --script=html5.config.rb
Standalone
This basically creates a custom scanner.
The difference is that these scripts will run a scan and handle its results on their own, and not just serve as configuration.
With helpers
When a scan script is large-ish and/or complicated it’s better to split it into the main file and helper handler methods.
bin/scnr_script scanner.rb
scanner.rb:
require 'scnr/engine/api'
require "#{Options.paths.root}/tmp/scripts/with_helpers/helpers"
SCNR::Application::API.run do
Scan {
# Can also be written as:
#
# options.set(
# url: 'http://testhtml5.vulnweb.com',
# audit: {
# elements: [:links, :forms, :cookies, :ui_inputs, :ui_forms]
# },
# checks: ['*']
# )
Options {
set url: 'http://my-site.com',
audit: {
elements: [:links, :forms, :cookies, :ui_inputs, :ui_forms]
},
checks: ['*']
}
# Scan session configuration.
Session {
# Login using the #fill_in_and_submit_the_login_form method from the helpers.rb file.
to :login, :fill_in_and_submit_the_login_form
# Check for a valid session using the #find_welcome_message method from the helpers.rb file.
to :check, :find_welcome_message
}
# Scan scope configuration.
Scope {
# Limit the scope of the scan based on URL.
select :url, :within_the_eshop
# Limit the scope of the scan based on Element.
reject :element, :with_sensitive_action; also :with_weird_nonce
# Only select pages that are in the admin panel.
select :page, :in_admin_panel
# Limit the scope of the scan based on Page.
reject :page, :with_error
# Limit the scope of the scan based on DOM events and DOM elements.
# In this case, never click the logout button!
reject :event, :that_clicks_the_logout_button
}
# Run the scan and handle the results (in this case print to STDOUT) using #handle_results.
run! :handle_results
}
Logging {
# Error and exception handling.
on :error, :log_error
on :exception, :log_exception
}
Data {
# Don't store issues in memory, we'll send them to the DB.
issues.disable(:storage).on :new, :save_to_db
# Could also be written as:
#
# Issues {
# disable(:storage)
# on :new, :save_to_db)
# }
#
# Or:
#
# Issues { disable(:storage); on :new, :save_to_db) }
# Store every page in the DB too for later analysis.
pages.on :new, :save_to_db
# Or:
#
# Pages {
# on :new, :save_to_db
# }
}
Http {
on :request, :add_special_auth_header
on :response, :gather_traffic_data; also :increment_http_performer_count
}
Checks {
# Add a custom check on the fly to check for something simple specifically
# for this scan.
as :missing_important_header, with_missing_important_header_info,
:log_pages_with_missing_important_headers
}
# Been having trouble with this scan, collect some runtime statistics.
plugins.as :remote_debug, send_debugging_info_to_remote_server_info,
:send_debugging_info_to_remote_server
# Serves PHP scripts under the extension 'x'.
fingerprinters.as :php_x, :treat_x_as_php
Input {
# Vouchers and serial numbers need to come from an algorithm.
values :with_valid_role_id
}
Dom {
# Let's have a look inside the live JS env of those interesting pages,
# setup the data collection.
before :load, :start_js_data_gathering
after :load, :retrieve_js_data; also :event, :retrieve_event_js_data
}
end
helpers.rb:
# State
def log_error( error )
# ...
end
def log_exception( exception )
# ...
end
# Data
def save_to_db( obj )
# Do stufff...
end
def save_js_data_to_db( data, element, event )
# Do other stufff...
end
# Scope
def within_the_eshop( url )
url.path.start_with? '/eshop'
end
def with_error( page )
/Error/i.match? page.body
end
def in_admin_panel( page )
/Admin panel/i.match? page.body
end
def that_clicks_the_logout_button( event, element )
event == :click && element.tag_name == :button &&
element.attributes['id'] == 'logout'
end
def with_sensitive_action( element )
element.action.include? '/sensitive.php'
end
def with_weird_nonce( element )
element.inputs.include? 'weird_nonce'
end
# HTTP
def generate_request_header
# ...
end
def save_raw_http_response( response )
# ...
end
def save_raw_http_request( request )
# ...
end
def add_special_auth_header( request )
request.headers['Special-Auth-Header'] ||= generate_request_header
end
def increment_http_performer_count( response )
# Count the amount of requests/responses this system component has
# performed/received.
#
# Performers can be browsers, checks, plugins, session, etc.
stuff( response.request.performer.class )
end
def gather_traffic_data( response )
# Collect raw HTTP traffic data.
save_raw_http_response( response.to_s )
save_raw_http_request( response.request.to_s )
end
# Checks
def with_missing_important_header_info
{
name: 'Missing Important-Header',
description: %q{Checks pages for missing `Important-Header` headers.},
elements: [ Element::Server ],
issue: {
name: %q{Missing 'Important-Header' header},
severity: Severity::INFORMATIONAL
}
}
end
# This will run from the context of a Check::Base.
def log_pages_with_missing_important_headers
return if audited?( page.parsed_url.host ) ||
page.response.headers['Important-Header']
audited( page.parsed_url.host )
log(
vector: Element::Server.new( page.url ),
proof: page.response.headers_string
)
end
# Plugins
# This will run from the context of a Plugin::Base.
def send_debugging_info_to_remote_server
address = '192.168.0.11'
port = 81
auth = Utilities.random_seed
url = `start_remote_debug_server.sh -a #{address} -p #{port} --auth #{auth}`
url.strip!
http.post( url,
body: SCNR::Engine::SCNR::Engine::Options.to_h.to_json,
mode: :sync
)
while framework.running? && sleep( 5 )
http.post( "#{url}/statistics",
body: framework.statistics.to_json,
mode: :sync
)
end
end
def send_debugging_info_to_remote_server_info
{
name: 'Debugger'
}
end
# Fingerprinters
# This will run from the context of a Fingerprinter::Base.
def treat_x_as_php
return if extension != 'x'
platforms << :php
end
# Session
def fill_in_and_submit_the_login_form( browser )
browser.load "#{SCNR::Engine::SCNR::Engine::Options.url}/login"
form = browser.form
form.text_field( name: 'username' ).set 'john'
form.text_field( name: 'password' ).set 'doe'
form.input( name: 'submit' ).click
end
def find_welcome_message
http.get( SCNR::Engine::Options.url, mode: :sync ).body.include?( 'Welcome user!' )
end
# Inputs
def with_valid_code( name, current_value )
{
'voucher-code' => voucher_code_generator( current_value ),
'serial-number' => serial_number_generator( current_value )
}[name]
end
def with_valid_role_id( inputs )
return if !inputs.include?( 'role-type' )
inputs['role-id'] ||= (inputs['role-type'] == 'manager' ? 1 : 2)
inputs
end
# Browser
def start_js_data_gathering( page, browser )
return if !page.url.include?( 'something/interesting' )
browser.javascript.inject <<JS
// Gather JS data from listeners etc.
window.secretJSData = {};
JS
end
def retrieve_js_data( page, browser )
return if !page.url.include?( 'something/interesting' )
save_js_data_to_db(
browser.javascript.run( 'return window.secretJSData' ),
page, :load
)
end
def retrieve_event_js_data( event, element, browser )
return if !browser.url.include?( 'something/interesting' )
save_js_data_to_db(
browser.javascript.run( 'return window.secretJSData' ),
element, event
)
end
def handle_results( report, statistics )
puts
puts '=' * 80
puts
puts "[#{report.sitemap.size}] Sitemap:"
puts
report.sitemap.sort_by { |url, _| url }.each do |url, code|
puts "\t[#{code}] #{url}"
end
puts
puts '-' * 80
puts
puts "[#{report.issues.size}] Issues:"
puts
report.issues.each.with_index do |issue, idx|
s = "\t[#{idx+1}] #{issue.name} in `#{issue.vector.type}`"
if issue.vector.respond_to?( :affected_input_name ) &&
issue.vector.affected_input_name
s << " input `#{issue.vector.affected_input_name}`"
end
puts s << '.'
puts "\t\tAt `#{issue.page.dom.url}` from `#{issue.referring_page.dom.url}`."
if issue.proof
puts "\t\tProof:\n\t\t\t#{issue.proof.gsub( "\n", "\n\t\t\t" )}"
end
puts
end
puts
puts '-' * 80
puts
puts "Statistics:"
puts
puts "\t" << statistics.ai.gsub( "\n", "\n\t" )
end
Single file
require 'scnr/engine/api'
# Mute output messages from the CLI interface, we've got our own output methods.
SCNR::UI::CLI::Output.mute
SCNR::Application::API.run do
State {
on :change do |state|
puts "State\t\t- #{state.status.capitalize}"
end
}
Data {
Issues {
on :new do |issue|
puts "Issue\t\t- #{issue.name} from `#{issue.referring_page.dom.url}`" <<
" in `#{issue.vector.type}`."
end
}
}
Logging {
on :error do |error|
$stderr.puts "Error\t\t- #{error}"
end
# Way too much noise.
# on :exception do |exception|
# ap exception
# ap exception.backtrace
# end
}
Dom {
# Allow some time for the modal animation to complete in order for
# the login form to appear.
#
# (Not actually necessary, this is just an example on how to hande quirks.)
on :event do |_, locator, event, *|
next if locator.attributes['href'] != '#myModal' || event != :click
sleep 1
end
}
Checks {
# This will run from the context of SCNR::Engine::Check::Base; it
# basically creates a new check component on the fly.
#
# Does something really simple, logs an issue for each 404 page.
as :not_found,
issue: {
name: 'Page not found',
severity: SCNR::Engine::Issue::Severity::INFORMATIONAL
} do
response = page.response
next if response.code != 404
log(
proof: response.status_line,
vector: SCNR::Engine::Element::Server.new( response.url ),
response: response
)
end
}
Plugins {
# This will run from the context of SCNR::Engine::Plugin::Base; it
# basically creates a new plugin component on the fly.
as :my_plugin do
puts "#{shortname}\t- Running..."
wait_while_framework_running
puts "#{shortname}\t- Done!"
end
}
Scan {
Options {
set url: 'http://testhtml5.vulnweb.com',
audit: {
elements: [:links, :forms, :cookies]
},
checks: ['*']
}
Session {
to :login do |browser|
print "Session\t\t- Logging in..."
# Login with whichever interface you prefer.
watir = browser.watir
selenium = browser.selenium
watir.goto SCNR::Engine::Options.url
watir.link( href: '#myModal' ).click
form = watir.form( id: 'loginForm' )
form.text_field( name: 'username' ).set 'admin'
form.text_field( name: 'password' ).set 'admin'
form.submit
if browser.response.body =~ /<b>admin/
puts 'done!'
else
puts 'failed!'
end
end
to :check do |async|
print "Session\t\t- Checking..."
http_client = SCNR::Engine::HTTP::Client
check = proc { |r| r.body.optimized_include? '<b>admin' }
# If an async block is passed, then the framework would rather
# schedule it to run asynchronously.
if async
http_client.get SCNR::Engine::Options.url do |response|
success = check.call( response )
puts "logged #{success ? 'in' : 'out'}!"
async.call success
end
else
response = http_client.get( SCNR::Engine::Options.url, mode: :sync )
success = check.call( response )
puts "logged #{success ? 'in' : 'out'}!"
success
end
end
}
Scope {
# Don't visit resources that will end the session.
reject :url do |url|
url.path.optimized_include?( 'login' ) ||
url.path.optimized_include?( 'logout' )
end
}
before :page do |page|
puts "Processing\t- [#{page.response.code}] #{page.dom.url}"
end
on :page do |page|
puts "Scanning\t- [#{page.response.code}] #{page.dom.url}"
end
after :page do |page|
puts "Scanned\t\t- [#{page.response.code}] #{page.dom.url}"
end
run! do |report, statistics|
puts
puts '=' * 80
puts
puts "[#{report.sitemap.size}] Sitemap:"
puts
report.sitemap.sort_by { |url, _| url }.each do |url, code|
puts "\t[#{code}] #{url}"
end
puts
puts '-' * 80
puts
puts "[#{report.issues.size}] Issues:"
puts
report.issues.each.with_index do |issue, idx|
s = "\t[#{idx+1}] #{issue.name} in `#{issue.vector.type}`"
if issue.vector.respond_to?( :affected_input_name ) &&
issue.vector.affected_input_name
s << " input `#{issue.vector.affected_input_name}`"
end
puts s << '.'
puts "\t\tAt `#{issue.page.dom.url}` from `#{issue.referring_page.dom.url}`."
if issue.proof
puts "\t\tProof:\n\t\t\t#{issue.proof.gsub( "\n", "\n\t\t\t" )}"
end
puts
end
puts
puts '-' * 80
puts
puts "Statistics:"
puts
puts "\t" << statistics.ai.gsub( "\n", "\n\t" )
end
}
end
Supposing the above is saved as html5.scanner.rb:
bin/scnr_script html5.scanner.rb
Distributed
Distribution features are deferred to Cuboid; hence, a quick read through its readme will outline the architecture.
In this case, the Cuboid application is SCNR.
Agent
To start an Agent run the scnr_agent CLI executable.
To see all available options run:
bin/scnr_agent -h
Each Agent should run on a different machine and its main role is to provide Instances to clients; each Instance is a scanner process.
The Agent will also split the available resources of the machine on which it runs into slots, with each slot corresponding to enough space for one Instance.
(To see how many slots a machine has you can use the scnr_system_info utility.)
Example
Server
In one terminal run:
bin/scnr_agent
The default port at the time of writing is 7331, so you should see something like:
I, [2022-01-23T09:54:21.849679 #1121060] INFO -- System: RPC Server started.
I, [2022-01-23T09:54:21.849730 #1121060] INFO -- System: Listening on 127.0.0.1:7331
Client
To start a scan originating from that Agent you must issue a spawn
call in order to obtain an Instance; this can be achieved using the scnr_spawn
CLI executable.
In another terminal run:
bin/scnr_spawn --agent-url=127.0.0.1:7331 http://testhtml5.vulnweb.com
The above will run a scan with the default options against http://testhtml5.vulnweb.com, originating from the Agent node.
The scnr_spawn utility largely accepts the same options as scnr.
If the Agent is out of slots you will see the following message:
[~] Agent is at maximum utilization, please try again later.
In which case you can keep retrying until a slot opens up.
Grid
A Grid is simply a group of Agents and its setup is as simple as specifying an already running Agent as a peer to a future Agent.
The order in which you start or specify peers is irrelevant, Agents will reach convergence on their own and keep track of their connectivity status with each other.
After a Grid is configured, when a spawn call is issued to any Grid
member it will be served by any of its Agents based on the desired
distribution strategy and not necessarily by the one receiving it.
Strategies
Horizontal (default)
spawn calls will be served by the least burdened Agent, i.e. the
Agent with the least utilization of its slots.
This strategy helps to keep the overall Grid health good by spreading the workload across as many nodes as possible.
Vertical
spawn calls will be served by the most burdened Agent, i.e. the
Agent with the most utilization of its slots.
This strategy helps to keep the overall Grid size (and thus cost) low by utilizing as few Grid nodes as possible.
It will also let you know if you have over-provisioned as extra nodes will not be receiving any workload.
Examples
Server
In one terminal run:
bin/scnr_agent
In another terminal run:
bin/scnr_agent --port=7332 --peer=127.0.0.1:7331
In another terminal run:
bin/scnr_agent --port=7333 --peer=127.0.0.1:7332
(It doesn’t matter who the peer is as long as it’s part of the Grid.)
Now we have a Grid of 3 Agents.
The point of course is to run each Agent on a different machine in real life.
Client
Same as Agent client.
Scheduler
To start a Scheduler run the scnr_scheduler CLI executable.
To see all available options run:
bin/scnr_scheduler -h
The main role of the Scheduler is to:
- Queue scans based on their assigned priority.
- Run them if there is an available slot.
- Monitor their progress.
- Grab and store reports once scans complete.
Default
By default, scans will run on the same machine as the Scheduler.
With Agent
When a Agent has been provided, spawn calls are going to be issued
in order to acquire Instances to run the scans.
Grid
In the case where the given Agent is a Grid member, scans will be load-balanced across the Grid according the the configured strategy.
Examples
Server
In one terminal run:
bin/scnr_scheduler
Client
Pushing
In another terminal run:
bin/scnr_scheduler_push --scheduler-url=localhost:7331 http://testhtml5.vulnweb.com
Then you should see something like:
[~] Pushed scan with ID: 5fed6c50f3699bacb841cc468cc97094
Monitoring
To see what the Scheduler is doing run:
bin/scnr_scheduler_list localhost:7331
Then you should see something like:
[~] Queued [0]
[*] Running [1]
[1] 5fed6c50f3699bacb841cc468cc97094: 127.0.0.1:3390/070116f5e2c0acaa0a6432acdcc7230a
[+] Completed [0]
[-] Failed [0]
If you run the same command after a while and the scan has completed:
[~] Queued [0]
[*] Running [0]
[+] Completed [1]
[1] 5fed6c50f3699bacb841cc468cc97094: /home/username/.cuboid/reports/5fed6c50f3699bacb841cc468cc97094.crf
[-] Failed [0]
IAST
Ruby
Add the following to your Gemfile:
gem "scnr-introspector"
Use the Rack Middleware
require 'scnr/introspector'
use SCNR::Introspector, scope: {
path_start_with: __FILE__
}
Example Sinatra app:
require 'scnr/introspector'
require 'sinatra/base'
class MyApp < Sinatra::Base
use SCNR::Introspector, scope: {
path_start_with: __FILE__
}
def noop
end
def process_params( params )
noop
params.values.join( ' ' )
end
get '/' do
@instance_variable = {
blah: 'foo'
}
local_variable = 1
<<EOHTML
#{process_params( params )}
<a href="?v=stuff">XSS</a>
EOHTML
end
run!
end
Scan as usual
./bin/scnr http://my-app/
Results
With IAST enabled on the server-side, the results will include data and execution flow trace information:
[+] 1 issues were detected.
[+] [1] Cross-Site Scripting (XSS) (Trusted)
[~] ~~~~~~~~~~~~~~~~~~~~
[~] Digest: 3187004085
[~] Severity: High
[~] Description:
[~]
Client-side scripts are used extensively by modern web applications.
They perform from simple functions (such as the formatting of text) up to full
manipulation of client-side data and Operating System interaction.
Cross Site Scripting (XSS) allows clients to inject scripts into a request and
have the server return the script to the client in the response. This occurs
because the application is taking untrusted data (in this example, from the client)
and reusing it without performing any validation or sanitisation.
If the injected script is returned immediately this is known as body XSS.
If the injected script is stored by the server and returned to any client visiting
the affected page, then this is known as persistent XSS (also stored XSS).
SCNR::Engine has discovered that it is possible to insert script content directly into
HTML element content.
[~] Tags: xss, regexp, injection, script
[~] CWE: http://cwe.mitre.org/data/definitions/79.html
[~] References:
[~] Secunia - http://secunia.com/advisories/9716/
[~] WASC - http://projects.webappsec.org/w/page/13246920/Cross%20Site%20Scripting
[~] OWASP - https://www.owasp.org/index.php/XSS_%28Cross_Site_Scripting%29_Prevention_Cheat_Sheet
[~] URL: http://localhost:4567/
[~] Element: link
[~] All inputs: v
[~] Method: GET
[~] Input name: v
[~] Seed: "<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
[~] Injected: "stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
[~] Proof: "<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
[~] Execution trace
[0] examples/sinatra/app.rb#17 MyApp#GET / call
[1] examples/sinatra/app.rb#17 MyApp#GET / b_call
[2] examples/sinatra/app.rb#19 MyApp#GET / line
[3] examples/sinatra/app.rb#21 MyApp#GET / line
[4] examples/sinatra/app.rb#24 MyApp#GET / line
[5] examples/sinatra/app.rb#12 MyApp#process_params call
[6] examples/sinatra/app.rb#13 MyApp#process_params line
[7] examples/sinatra/app.rb#9 MyApp#noop call
[8] examples/sinatra/app.rb#10 MyApp#noop return
[9] examples/sinatra/app.rb#14 MyApp#process_params line
[10] examples/sinatra/app.rb#14 Hash#values c_call
[11] examples/sinatra/app.rb#14 Hash#values c_return
[12] examples/sinatra/app.rb#14 Array#join c_call
[13] examples/sinatra/app.rb#14 Array#join c_return
[14] examples/sinatra/app.rb#15 MyApp#process_params return
[15] examples/sinatra/app.rb#27 MyApp#GET / b_return
[~] Data trace
[0] MyApp#call argument #0: "v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E"
Arguments:
[
{
"rack.version": [
1,
6
],
"rack.errors": "#<IO:0x000055d59b7eb758>",
"rack.multithread": true,
"rack.multiprocess": false,
"rack.run_once": false,
"rack.url_scheme": "http",
"SCRIPT_NAME": "",
"QUERY_STRING": "v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E",
"SERVER_SOFTWARE": "puma 6.2.2 Speaking of Now",
"GATEWAY_INTERFACE": "CGI/1.2",
"REQUEST_METHOD": "GET",
"REQUEST_PATH": "/",
"REQUEST_URI": "/?v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E",
"SERVER_PROTOCOL": "HTTP/1.1",
"HTTP_HOST": "localhost:4567",
"HTTP_ACCEPT_ENCODING": "gzip, deflate",
"HTTP_USER_AGENT": "Mozilla/5.0 (Gecko) SCNR::Engine/v0.1.2",
"HTTP_ACCEPT": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"HTTP_ACCEPT_LANGUAGE": "en-US,en;q=0.8,he;q=0.6",
"puma.request_body_wait": 0.006314277648925781,
"SERVER_NAME": "localhost",
"SERVER_PORT": "4567",
"PATH_INFO": "/",
"REMOTE_ADDR": "127.0.0.1",
"HTTP_VERSION": "HTTP/1.1",
"puma.socket": "#<TCPSocket:0x000055d59c8b2078>",
"rack.hijack?": true,
"rack.hijack": "#<Puma::Client:0x000055d59c8b2028>",
"rack.input": "#<Puma::NullIO:0x000055d59c4b2c20>",
"rack.after_reply": [
],
"puma.config": "#<Puma::Configuration:0x000055d59c1379d8>",
"rack.logger": "#<Rack::NullLogger:0x000055d59bdbe7c0>",
"rack.request.query_string": "v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E",
"rack.request.query_hash": {
"v": "stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
},
"sinatra.route": "GET /"
}
]
Backtrace:
(eval):4:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/xss_header.rb:20:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/path_traversal.rb:18:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/json_csrf.rb:28:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/frame_options.rb:33:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/null_logger.rb:11:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/head.rb:12:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:219:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:2018:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `block in call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1792:in `synchronize'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/configuration.rb:270:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:98:in `block in handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:97:in `handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:431:in `process_client'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:233:in `block in run'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
[1] MyApp#call! argument #0: "v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E"
Arguments:
[
{
"rack.version": [
1,
6
],
"rack.errors": "#<IO:0x000055d59b7eb758>",
"rack.multithread": true,
"rack.multiprocess": false,
"rack.run_once": false,
"rack.url_scheme": "http",
"SCRIPT_NAME": "",
"QUERY_STRING": "v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E",
"SERVER_SOFTWARE": "puma 6.2.2 Speaking of Now",
"GATEWAY_INTERFACE": "CGI/1.2",
"REQUEST_METHOD": "GET",
"REQUEST_PATH": "/",
"REQUEST_URI": "/?v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E",
"SERVER_PROTOCOL": "HTTP/1.1",
"HTTP_HOST": "localhost:4567",
"HTTP_ACCEPT_ENCODING": "gzip, deflate",
"HTTP_USER_AGENT": "Mozilla/5.0 (Gecko) SCNR::Engine/v0.1.2",
"HTTP_ACCEPT": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"HTTP_ACCEPT_LANGUAGE": "en-US,en;q=0.8,he;q=0.6",
"puma.request_body_wait": 0.006314277648925781,
"SERVER_NAME": "localhost",
"SERVER_PORT": "4567",
"PATH_INFO": "/",
"REMOTE_ADDR": "127.0.0.1",
"HTTP_VERSION": "HTTP/1.1",
"puma.socket": "#<TCPSocket:0x000055d59c8b2078>",
"rack.hijack?": true,
"rack.hijack": "#<Puma::Client:0x000055d59c8b2028>",
"rack.input": "#<Puma::NullIO:0x000055d59c4b2c20>",
"rack.after_reply": [
],
"puma.config": "#<Puma::Configuration:0x000055d59c1379d8>",
"rack.logger": "#<Rack::NullLogger:0x000055d59bdbe7c0>",
"rack.request.query_string": "v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E",
"rack.request.query_hash": {
"v": "stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
},
"sinatra.route": "GET /"
}
]
Backtrace:
(eval):4:in `call!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:938:in `call'
(eval):5:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/xss_header.rb:20:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/path_traversal.rb:18:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/json_csrf.rb:28:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/frame_options.rb:33:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/null_logger.rb:11:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/head.rb:12:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:219:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:2018:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `block in call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1792:in `synchronize'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/configuration.rb:270:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:98:in `block in handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:97:in `handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:431:in `process_client'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:233:in `block in run'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
[2] MyApp#process_params argument #0: "stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
Arguments:
[
{
"v": "stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>"
}
]
Backtrace:
(eval):4:in `process_params'
examples/sinatra/app.rb:24:in `block in <class:MyApp>'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1706:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1706:in `block in compile!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1019:in `block (3 levels) in route!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1037:in `route_eval'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1019:in `block (2 levels) in route!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1068:in `block in process_route'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1066:in `catch'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1066:in `process_route'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1017:in `block in route!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1014:in `each'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1014:in `route!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1138:in `block in dispatch!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1109:in `catch'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1109:in `invoke'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1133:in `dispatch!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:949:in `block in call!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1109:in `catch'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1109:in `invoke'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:949:in `call!'
(eval):5:in `call!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:938:in `call'
(eval):5:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/xss_header.rb:20:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/path_traversal.rb:18:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/json_csrf.rb:28:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/frame_options.rb:33:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/null_logger.rb:11:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/head.rb:12:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:219:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:2018:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `block in call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1792:in `synchronize'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/configuration.rb:270:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:98:in `block in handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:97:in `handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:431:in `process_client'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:233:in `block in run'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
[3] MyApp#body argument #0: "stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>\n <a href=\"?v=stuff\">XSS</a>\n"
Arguments:
[
[
"stuff<xss_4e12ad53210ef9db2fe43d2bd73eee80/>\n <a href=\"?v=stuff\">XSS</a>\n"
]
]
Backtrace:
(eval):4:in `body'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1118:in `invoke'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1133:in `dispatch!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:949:in `block in call!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1109:in `catch'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1109:in `invoke'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:949:in `call!'
(eval):5:in `call!'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:938:in `call'
(eval):5:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/xss_header.rb:20:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/path_traversal.rb:18:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/json_csrf.rb:28:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/base.rb:53:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-protection-3.0.6/lib/rack/protection/frame_options.rb:33:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/null_logger.rb:11:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/rack-2.2.7/lib/rack/head.rb:12:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:219:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:2018:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `block in call'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1792:in `synchronize'
/home/zapotek/scnr-dev-env/.system/gems/gems/sinatra-3.0.6/lib/sinatra/base.rb:1576:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/configuration.rb:270:in `call'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:98:in `block in handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/request.rb:97:in `handle_request'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:431:in `process_client'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/server.rb:233:in `block in run'
/home/zapotek/scnr-dev-env/.system/gems/gems/puma-6.2.2/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
[~] Referring page: http://localhost:4567/
[~] Affected page: http://localhost:4567/?v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80/%3E
[~] HTTP request
GET /?v=stuff%3Cxss_4e12ad53210ef9db2fe43d2bd73eee80%2F%3E HTTP/1.1
Host: localhost:4567
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Gecko) SCNR::Engine/v0.1.2
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.8,he;q=0.6
X-Scnr-Engine-Scan-Seed: 4e12ad53210ef9db2fe43d2bd73eee80
X-Scnr-Introspector-Taint: 4e12ad53210ef9db2fe43d2bd73eee80
X-Scnr-Introspector-Trace: 2
CLI
Command-line interface executables can be found under the bin/ directory and
at the time of writing are:
scnr– Direct scanning utility.scnr_console– A REPL Ruby console running from the context ofSCNR::Engine.scnr_spawn– Issuesspawncalls to Agents to start scans remotely.scnr_agent– Starts a Agent daemon.scnr_agent_monitor– Monitors a Agent.scnr_agent_unplug– Unplugs a Agent from its Grid.scnr_instance_connect– Utility to connect to an Instance.scnr_reporter– Generates reports from.crf(Cuboid report file) and.ser(SCNR Engine report) report files.scnr_reproduce– Reproduces an issue(s) from a given report.scnr_rest_server– Starts a REST server daemon.scnr_restore– Restores a suspended scan based on a snapshot file.scnr_scheduler– Starts a Scheduler daemon.scnr_scheduler_attach– Attaches a detached Instance to the given Scheduler.scnr_scheduler_clear– Clears the Scheduler queue.scnr_scheduler_detach– Detaches an Instance from the Scheduler.scnr_scheduler_get– Retrieves information for a scheduled scan.scnr_scheduler_list– Lists information about all scans under the Scheduler’s control.scnr_scheduler_push– Scheduled a scan.scnr_scheduler_remove– Removes a scheduled scan from the queue.scnr_script– Runs a Ruby script under the context ofSCNR::Engine.scnr_shell– Starts a Bash shell under the package environment.scnr_system_info– Presents system information about the host.
Ruby API
The Ruby API utilizes the DSeL DSL/API generator and runner and allows you to:
- Configure scans.
- Add custom components on the fly.
- Create custom scanners.
The API is separated into the following segments:
# Runs the DSL.
SCNR::Application::API.run do
Data {
Sitemap {}
Urls {}
Pages {}
Issues {}
}
State { }
Browserpool { }
Dom { }
Http { }
Input { }
Logging { }
Checks { }
Plugins { }
Fingerprinters { }
Scan {
Options { }
Scope { }
Session { }
}
end
Examples
As configuration
SCNR::Application::API.run do
Dom {
# Allow some time for the modal animation to complete in order for
# the login form to appear.
#
# (Not actually necessary, this is just an example on how to hande quirks.)
on :event do |_, locator, event, *|
next if locator.attributes['href'] != '#myModal' || event != :click
sleep 1
end
}
Checks {
# This will run from the context of SCNR::Engine::Check::Base; it
# basically creates a new check component on the fly.
#
# Does something really simple, logs an issue for each 404 page.
as :not_found,
issue: {
name: 'Page not found',
severity: SCNR::Engine::Issue::Severity::INFORMATIONAL
} do
response = page.response
next if response.code != 404
log(
proof: response.status_line,
vector: SCNR::Engine::Element::Server.new( response.url ),
response: response
)
end
}
Plugins {
# This will run from the context of SCNR::Engine::Plugin::Base; it
# basically creates a new plugin component on the fly.
as :my_plugin do
# Do stuff then wait until scan completes.
wait_while_framework_running
# Do stuff after scan completes.
end
}
Scan {
Session {
to :login do |browser|
# Login with whichever interface you prefer.
watir = browser.watir
selenium = browser.selenium
watir.goto SCNR::Engine::Options.url
watir.link( href: '#myModal' ).click
form = watir.form( id: 'loginForm' )
form.text_field( name: 'username' ).set 'admin'
form.text_field( name: 'password' ).set 'admin'
form.submit
end
to :check do |async|
http_client = SCNR::Engine::HTTP::Client
check = proc { |r| r.body.optimized_include? '<b>admin' }
# If an async block is passed, then the framework would rather
# schedule it to run asynchronously.
if async
http_client.get SCNR::Engine::Options.url do |response|
success = check.call( response )
async.call success
end
else
response = http_client.get( SCNR::Engine::Options.url, mode: :sync )
check.call( response )
end
end
}
Scope {
# Don't visit resources that will end the session.
reject :url do |url|
url.path.optimized_include?( 'login' ) ||
url.path.optimized_include?( 'logout' )
end
}
}
end
Supposing the above is saved as html5.config.rb:
bin/scnr http://testhtml5.vulnweb.com --script=html5.config.rb
Standalone
This basically creates a custom scanner.
require 'scnr/engine/api'
# Mute output messages from the CLI interface, we've got our own output methods.
SCNR::UI::CLI::Output.mute
SCNR::Application::API.run do
State {
on :change do |state|
puts "State\t\t- #{state.status.capitalize}"
end
}
Data {
Issues {
on :new do |issue|
puts "Issue\t\t- #{issue.name} from `#{issue.referring_page.dom.url}`" <<
" in `#{issue.vector.type}`."
end
}
}
Logging {
on :error do |error|
$stderr.puts "Error\t\t- #{error}"
end
# Way too much noise.
# on :exception do |exception|
# ap exception
# ap exception.backtrace
# end
}
Dom {
# Allow some time for the modal animation to complete in order for
# the login form to appear.
#
# (Not actually necessary, this is just an example on how to hande quirks.)
on :event do |_, locator, event, *|
next if locator.attributes['href'] != '#myModal' || event != :click
sleep 1
end
}
Checks {
# This will run from the context of SCNR::Engine::Check::Base; it
# basically creates a new check component on the fly.
#
# Does something really simple, logs an issue for each 404 page.
as :not_found,
issue: {
name: 'Page not found',
severity: SCNR::Engine::Issue::Severity::INFORMATIONAL
} do
response = page.response
next if response.code != 404
log(
proof: response.status_line,
vector: SCNR::Engine::Element::Server.new( response.url ),
response: response
)
end
}
Plugins {
# This will run from the context of SCNR::Engine::Plugin::Base; it
# basically creates a new plugin component on the fly.
as :my_plugin do
puts "#{shortname}\t- Running..."
wait_while_framework_running
puts "#{shortname}\t- Done!"
end
}
Scan {
Options {
set url: 'http://testhtml5.vulnweb.com',
audit: {
elements: [:links, :forms, :cookies]
},
checks: ['*']
}
Session {
to :login do |browser|
print "Session\t\t- Logging in..."
# Login with whichever interface you prefer.
watir = browser.watir
selenium = browser.selenium
watir.goto SCNR::Engine::Options.url
watir.link( href: '#myModal' ).click
form = watir.form( id: 'loginForm' )
form.text_field( name: 'username' ).set 'admin'
form.text_field( name: 'password' ).set 'admin'
form.submit
if browser.response.body =~ /<b>admin/
puts 'done!'
else
puts 'failed!'
end
end
to :check do |async|
print "Session\t\t- Checking..."
http_client = SCNR::Engine::HTTP::Client
check = proc { |r| r.body.optimized_include? '<b>admin' }
# If an async block is passed, then the framework would rather
# schedule it to run asynchronously.
if async
http_client.get SCNR::Engine::Options.url do |response|
success = check.call( response )
puts "logged #{success ? 'in' : 'out'}!"
async.call success
end
else
response = http_client.get( SCNR::Engine::Options.url, mode: :sync )
success = check.call( response )
puts "logged #{success ? 'in' : 'out'}!"
success
end
end
}
Scope {
# Don't visit resources that will end the session.
reject :url do |url|
url.path.optimized_include?( 'login' ) ||
url.path.optimized_include?( 'logout' )
end
}
before :page do |page|
puts "Processing\t- [#{page.response.code}] #{page.dom.url}"
end
on :page do |page|
puts "Scanning\t- [#{page.response.code}] #{page.dom.url}"
end
after :page do |page|
puts "Scanned\t\t- [#{page.response.code}] #{page.dom.url}"
end
run! do |report, statistics|
puts
puts '=' * 80
puts
puts "[#{report.sitemap.size}] Sitemap:"
puts
report.sitemap.sort_by { |url, _| url }.each do |url, code|
puts "\t[#{code}] #{url}"
end
puts
puts '-' * 80
puts
puts "[#{report.issues.size}] Issues:"
puts
report.issues.each.with_index do |issue, idx|
s = "\t[#{idx+1}] #{issue.name} in `#{issue.vector.type}`"
if issue.vector.respond_to?( :affected_input_name ) &&
issue.vector.affected_input_name
s << " input `#{issue.vector.affected_input_name}`"
end
puts s << '.'
puts "\t\tAt `#{issue.page.dom.url}` from `#{issue.referring_page.dom.url}`."
if issue.proof
puts "\t\tProof:\n\t\t\t#{issue.proof.gsub( "\n", "\n\t\t\t" )}"
end
puts
end
puts
puts '-' * 80
puts
puts "Statistics:"
puts
puts "\t" << statistics.ai.gsub( "\n", "\n\t" )
end
}
end
Supposing the above is saved as html5.scanner.rb:
bin/scnr_script html5.scanner.rb
Data
Encapsulates functionality that has to do with the data of SCNR::Engine.
SCNR::Application::API.run do
Data {
Sitemap {}
Urls {}
Pages {}
Issues {}
}
end
Sitemap
SCNR::Application::API.run do
Data {
Sitemap {
on :new do |entry|
p entry
# => { "http://example.com" => 200 }
# URL => HTTP code
end
}
}
end
Example
bin/scnr http://example.com/ --checks=- --script=sitemap.rb
Urls
SCNR::Application::API.run do
Data {
Urls {
on :new do |url|
p url
# => "http://example.com"
end
}
}
end
Example
bin/scnr http://example.com/ --checks=- --script=urls.rb
Pages
SCNR::Application::API.run do
Data {
Pages {
on :new do |page|
p page
# => #<SCNR::Engine::Page:7240 @url="http://testhtml5.vulnweb.com/ajax/popular?offset=0" @dom=#<SCNR::Engine::Page::DOM:7260 @url="http://testhtml5.vulnweb.com/ajax/popular?offset=0" @transitions=1 @data_flow_sinks=0 @execution_flow_sinks=0>>
end
}
}
end
Example
bin/scnr http://testhtml5.vulnweb.com/ --checks=- --script=pages.rb
Issues
SCNR::Application::API.run do
Data {
Issues {
on :new do |issue|
p issue
# => #<SCNR::Engine::Issue:0x00007f8c50d825a0 @name="Allowed HTTP methods", @description="\nThere are a number of HTTP methods that can be used on a webserver (`OPTIONS`,\n`HEAD`, `GET`, `POST`, `PUT`, `DELETE` etc.). Each of these methods perform a\ndifferent function and each have an associated level of risk when their use is\npermitted on the webserver.\n\nA client can use the `OPTIONS` method within a request to query a server to\ndetermine which methods are allowed.\n\nCyber-criminals will almost always perform this simple test as it will give a\nvery quick indication of any high-risk methods being permitted by the server.\n\nSCNR::Engine discovered that several methods are supported by the server.\n", @references={"Apache.org"=>"http://httpd.apache.org/docs/2.2/mod/core.html#limitexcept"}, @tags=["http", "methods", "options"], @severity=#<SCNR::Engine::Issue::Severity::Base:0x00007f8c50dccee8 @severity=:informational>, @remedy_guidance="\nIt is recommended that a whitelisting approach be taken to explicitly permit the\nHTTP methods required by the application and block all others.\n\nTypically the only HTTP methods required for most applications are `GET` and\n`POST`. All other methods perform actions that are rarely required or perform\nactions that are inherently risky.\n\nThese risky methods (such as `PUT`, `DELETE`, etc) should be protected by strict\nlimitations, such as ensuring that the channel is secure (SSL/TLS enabled) and\nonly authorised and trusted clients are permitted to use them.\n", @check={:name=>"Allowed methods", :description=>"Checks for supported HTTP methods.", :elements=>[SCNR::Engine::Element::Server], :cost=>1, :author=>"Tasos \"Zapotek\" Laskos <[email protected]>", :version=>"0.2", :shortname=>"allowed_methods"}, @vector=#<SCNR::Engine::Element::Server url="http://example.com/">, @proof="OPTIONS, GET, HEAD, POST", @referring_page=#<SCNR::Engine::Page:6560 @url="http://example.com/" @dom=#<SCNR::Engine::Page::DOM:6580 @url="http://example.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0>>, @platform_name=nil, @platform_type=nil, @page=#<SCNR::Engine::Page:6600 @url="http://example.com/" @dom=#<SCNR::Engine::Page::DOM:6620 @url="http://example.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0>>, @remarks={}, @trusted=true>
end
# Disables Issue storage.
disable :storage
}
}
end
Example
bin/scnr http://example.com/ --checks=allowed_methods --script=issues.rb
State
SCNR::Application::API.run do
State {
on :change do |state|
p state.status
# => :preparing
end
}
end
Example
bin/scnr http://example.com/ --checks=- --script=state.rb
Browserpool
SCNR::Application::API.run do
Browserpool {
# When a job is queued.
on :job do |job|
p job
# => #<SCNR::Engine::BrowserPool::Jobs::DOMExploration:6140 @resource=#<SCNR::Engine::Page::DOM:6160 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0> time= timed_out=false>
end
# When a job has completed.
on :job_done do |job|
p job
# => #<SCNR::Engine::BrowserPool::Jobs::DOMExploration:6140 @resource=#<SCNR::Engine::Page::DOM:6160 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0> time=2.805975399 timed_out=false>
end
# When a job has yielded a result.
on :result do |result|
p result
# => #<SCNR::Engine::BrowserPool::Jobs::DOMExploration::Result:0x00007f51a167c218 @page=#<SCNR::Engine::Page:7340 @url="http://testhtml5.vulnweb.com/ajax/popular?offset=0" @dom=#<SCNR::Engine::Page::DOM:7360 @url="http://testhtml5.vulnweb.com/ajax/popular?offset=0" @transitions=1 @data_flow_sinks=0 @execution_flow_sinks=0>>, @job=#<SCNR::Engine::BrowserPool::Jobs::DOMExploration:7320 @resource= time= timed_out=false>>
end
}
end
Example
bin/scnr http://html5.vulnweb.com/ --checks=- --script=browserpool.rb
Dom
SCNR::Application::API.run do
Dom {
before :load do |resource, options, browser|
p resource
# => #<SCNR::Engine::Page::DOM:6160 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0>
p options
# => {:take_snapshot=>true}
p browser
# => #<SCNR::Engine::BrowserPool::Worker pid= job=#<SCNR::Engine::BrowserPool::Jobs::DOMExploration:6140 @resource=#<SCNR::Engine::Page::DOM:6160 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0> time= timed_out=false> last-url=nil transitions=0>
end
before :event do |locator, event, options, browser|
p locator
# => <li class="active" id="popularLi">
p event
# => :click
p options
# => {}
p browser
# => #<SCNR::Engine::BrowserPool::Worker pid= job=#<SCNR::Engine::BrowserPool::Jobs::DOMExploration::EventTrigger:7760 @resource=#<SCNR::Engine::Page::DOM:7720 @url="http://testhtml5.vulnweb.com/#/popular" @transitions=17 @data_flow_sinks=0 @execution_flow_sinks=0> time= timed_out=false> last-url="http://testhtml5.vulnweb.com/" transitions=17>
end
on :event do |success, locator, event, options, browser|
p success
# => true
p locator
# => <li class="active" id="popularLi">
p event
# => :click
p options
# => {}
p browser
# => #<SCNR::Engine::BrowserPool::Worker pid= job=#<SCNR::Engine::BrowserPool::Jobs::DOMExploration::EventTrigger:7760 @resource=#<SCNR::Engine::Page::DOM:7720 @url="http://testhtml5.vulnweb.com/#/popular" @transitions=17 @data_flow_sinks=0 @execution_flow_sinks=0> time= timed_out=false> last-url="http://testhtml5.vulnweb.com/" transitions=17>
end
after :load do |resource, options, browser|
p resource
# => #<SCNR::Engine::Page::DOM:6160 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0>
p options
# => {:take_snapshot=>true}
p browser
# => #<SCNR::Engine::BrowserPool::Worker pid= job=#<SCNR::Engine::BrowserPool::Jobs::DOMExploration:6140 @resource=#<SCNR::Engine::Page::DOM:6160 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0> time= timed_out=false> last-url="http://testhtml5.vulnweb.com/" transitions=17>
end
after :event do |transition, locator, event, options, browser|
p transition
# => #<SCNR::Engine::Page::DOM::Transition:0x00007f50fc0739c0 @options={}, @event=:click, @element=<a data-scnr-engine-id="1270713017" href="#/popular">, @clock=nil, @time=0.036003384>
p locator
# => <a data-scnr-engine-id="1270713017" href="#/popular">
p event
# => :click
p options
# => {}
p browser
# => #<SCNR::Engine::BrowserPool::Worker pid= job=#<SCNR::Engine::BrowserPool::Jobs::DOMExploration::EventTrigger:7680 @resource=#<SCNR::Engine::Page::DOM:7620 @url="http://testhtml5.vulnweb.com/#/popular" @transitions=17 @data_flow_sinks=0 @execution_flow_sinks=0> time= timed_out=false> last-url="http://testhtml5.vulnweb.com/" transitions=17>
end
}
end
Example
bin/scnr http://html5.vulnweb.com/ --checks=- --script=dom.rb
Http
SCNR::Application::API.run do
Http {
on :request do |request|
p request
# => #<SCNR::Engine::HTTP::Request @id= @mode=async @method=get @url="https://wordpress.com/" @parameters={} @high_priority= @performer=#<SCNR::Engine::Framework (scanning) runtime=0.805773919 found-pages=0 audited-pages=0 issues=0 checks= plugins=autothrottle,healthmap,discovery,timing_attacks,uniformity>>
end
on :response do |response|
p response
# => #<SCNR::Engine::HTTP::Response:0x00007fd6e75923b8 ..>
end
on :cookies do |cookies|
p cookies
# => [#<SCNR::Engine::Element::Cookie (get) url="https://wordpress.com/start/?ref=logged-out-homepage-lp" action="https://wordpress.com/start/?ref=logged-out-homepage-lp" default-inputs={"country_code"=>"GR"} inputs={"country_code"=>"GR"} raw_inputs=[] >]
end
# Block to run after each HTTP request batch run.
after :run do
end
}
end
Example
bin/scnr https://wordpress.com --checks=- --script=http.rb
Input
SCNR::Application::API.run do
Input {
# Fill-in values for the given element; must return Hash not alter the element.
values do |element|
p element
# => #<SCNR::Engine::Element::Form (post) auditor=SCNR::Engine::Trainer::SinkTracer url="http://testhtml5.vulnweb.com/" action="http://testhtml5.vulnweb.com/login" default-inputs={"username"=>"admin", "password"=>"", "loginFormSubmit"=>""} inputs={"username"=>"admin", "password"=>"5543!%scnr_engine_secret", "loginFormSubmit"=>"1"} raw_inputs=[] >
element.inputs
end
}
end
Example
bin/scnr https://testhtml5.vulnweb.com --checks=xss --script=input.rb
Logging
SCNR::Application::API.run do
Logging {
# Will get called for each error message that is logged.
on :error do |error|
p error
# => "Error string"
end
# Will get called for each exception that is created, even if safely handled.
on :exception do |exception|
p exception
# => #<SCNR::Engine::URICommon::Error: Failed to parse URL.>
end
}
end
Example
bin/scnr https://example.com --checks=- --script=logging.rb
Checks
SCNR::Application::API.run do
Checks {
# Will get called for each check that is run.
on :run do |check|
p check
# => #<SCNR::Engine::Checks::BackupDirectories:0x00007f2d55c66bf8 @page=#<SCNR::Engine::Page:7920 @url="http://testhtml5.vulnweb.com/" @dom=#<SCNR::Engine::Page::DOM:7940 @url="http://testhtml5.vulnweb.com/" @transitions=0 @data_flow_sinks=0 @execution_flow_sinks=0>>>
end
# This will run from the context of SCNR::Engine::Check::Base; it
# basically creates a new check component on the fly.
#
# This one does something really simple, logs an issue for each 404 page.
as :not_found,
issue: {
name: 'Page not found',
severity: SCNR::Engine::Issue::Severity::INFORMATIONAL
} do
response = page.response
next if response.code != 404
log(
proof: response.status_line,
vector: SCNR::Engine::Element::Server.new( response.url ),
response: response
)
end
}
end
Example
bin/scnr http://testhtml5.vulnweb.com --script=checks.rb
Plugins
SCNR::Application::API.run do
Plugins {
# Will get called upon plugin class initialization.
on :initialize do |plugin|
p plugin
# => #<SCNR::Engine::Plugins::AutoThrottle:0x00007f8896ccacd8 @options={}>
end
# Will get called when each plugin's #prepare method is called.
on :prepare do |plugin|
end
# Will get called when each plugin's #run method is called.
on :run do |plugin|
end
# Will get called when each plugin's #clean_up method is called.
on :clean_up do |plugin|
end
# Will get called when each plugin is done running.
on :done do |plugin|
end
# This will run from the context of SCNR::Engine::Plugin::Base; it
# basically creates a new plugin component on the fly.
as :my_plugin do
# Do stuff then wait until scan completes.
wait_while_framework_running
# Do stuff after scan completes.
end
}
end
Example
bin/scnr http://testhtml5.vulnweb.com --checks=- --script=plugins.rb
Fingerprinters
SCNR::Application::API.run do
Fingerprinters {
# Identify `*.x` resources as PHP.
as :x_as_php do
next unless extension == 'x'
platforms << :php
end
}
end
Example
bin/scnr http://testhtml5.vulnweb.com --checks=- --script=fingerprinters.rb
Scan
Encapsulates functionality that has to do with the scan.
SCNR::Application::API.run do
Scan {
Options {}
Scope {}
Sesion {}
# Called before each page audit.
before :page do |page|
end
# Called on page audit.
on :page do |page|
end
# Called after a page audit.
after :page do |page|
end
# Perform the scan.
run! do |report, statistics|
end
# Perform the scan.
report, statistics = self.run
# Get scan progress.
progress = self.progress
# Get scan progress updates for session :my_session (any user-provided session ID will do).
progress = self.session_progress( :my_session )
sitemap = self.sitemap
status = self.status
issues = self.issues
statistics = self.statistics
is_running = self.running?
is_scanning = self.scanning?
# Pauses the scan.
self.pause!
# Resumes the scan.
self.resume!
# Aborts the scan.
self.abort!
# Suspends the scan.
self.suspend!
is_pausing = self.pausing?
is_paused = self.paused?
is_suspending = self.suspending?
is_suspended = self.suspended?
# Restores a scan.
self.restore!( snapshot_path )
# Get a scan report.
report = self.generate_report
}
end
Example
SCNR::UI::CLI::Output.mute
api = SCNR::Application::API.new
api.scan.options.set url: 'http://testhtml5.vulnweb.com',
checks: %w(allowed_methods interesting_responses)
api.state.on :change do |state|
puts "Status:"
ap state.status
end
api.data.sitemap.on :new do |entry|
puts "Sitemap entry:"
ap entry
end
api.data.issues.on :new do |issue|
puts "New issue:"
ap issue
end
scan_thread = Thread.new { api.scan.run }
while scan_thread.alive?
puts "Progress update:"
ap api.scan.session_progress( :session )
sleep 1
end
ap api.scan.generate_report
Assuming the above is saved as html5.scanner.rb:
bin/scnr_script html5.scanner.rb
Options
SCNR::Application::API.run do
Scan {
Options {
# Sets options.
set({
url: 'http://testhtml5.vulnweb.com',
audit: {
parameter_values: true,
paranoia: :medium,
exclude_vector_patterns: [],
include_vector_patterns: [],
link_templates: []
},
device: {
visible: false,
width: 1600,
height: 1200,
user_agent: "Mozilla/5.0 (Gecko) SCNR::Engine/v1.0dev",
pixel_ratio: 1.0,
touch: false
},
dom: {
engine: :chrome,
local_storage: {},
session_storage: {},
wait_for_elements: {},
pool_size: 4,
job_timeout: 60,
worker_time_to_live: 250,
wait_for_timers: false
},
http: {
request_timeout: 20000,
request_redirect_limit: 5,
request_concurrency: 10,
request_queue_size: 50,
request_headers: {},
response_max_size: 500000,
cookies: {},
authentication_type: "auto"
},
input: {
values: {},
default_values: {
"name" => "scnr_engine_name",
"user" => "scnr_engine_user",
"usr" => "scnr_engine_user",
"pass" => "5543!%scnr_engine_secret",
"txt" => "scnr_engine_text",
"num" => "132",
"amount" => "100",
"mail" => "[email protected]",
"account" => "12",
"id" => "1"
},
without_defaults: false,
force: false
},
scope: {
directory_depth_limit: 10,
auto_redundant_paths: 15,
redundant_path_patterns: {},
dom_depth_limit: 4,
dom_event_limit: 500,
dom_event_inheritance_limit: 500,
exclude_file_extensions: [],
exclude_path_patterns: [],
exclude_content_patterns: [],
include_path_patterns: [],
restrict_paths: [],
extend_paths: [],
url_rewrites: {}
},
session: {},
checks: [
"*"
],
platforms: [],
plugins: {},
no_fingerprinting: false,
authorized_by: nil
})
}
}
end
Scope
Determines which resources are in or out of scope. All return values will be cast to boolean.
SCNR::Application::API.run do
Scan {
Scope {
select :url do |url|
end
select :page do |page|
end
select :element do |element|
end
select :event do |locator, event, options, browser|
end
reject :url do |url|
end
reject :page do |page|
end
reject :element do |element|
end
reject :event do |locator, event, options, browser|
end
}
}
end
Example
bin/scnr http://testhtml5.vulnweb.com --checks=- --script=scope.rb
Session
SCNR::Application::API.run do
Scan {
Session {
to :login do |browser|
# Login with whichever interface you prefer.
watir = browser.watir
selenium = browser.selenium
watir.goto SCNR::Engine::Options.url
watir.link( href: '#myModal' ).click
form = watir.form( id: 'loginForm' )
form.text_field( name: 'username' ).set 'admin'
form.text_field( name: 'password' ).set 'admin'
form.submit
end
to :check do |async|
http_client = SCNR::Engine::HTTP::Client
check = proc { |r| r.body.optimized_include? '<b>admin' }
# If an async block is passed, then the framework would rather
# schedule it to run asynchronously.
if async
http_client.get SCNR::Engine::Options.url do |response|
success = check.call( response )
async.call success
end
else
response = http_client.get( SCNR::Engine::Options.url, mode: :sync )
check.call( response )
end
end
}
}
end
Example
bin/scnr http://testhtml5.vulnweb.com --checks=- --script=session.rb
REST
Server
To start the REST server daemon:
bin/scnr_rest_server
To see REST server daemon options:
bin/scnr_rest_server -h
API
Scan
| Method | Resource | Parameters | Description |
|---|---|---|---|
GET | /instances/:id/scan/progress | Get scan progress. | |
GET | /instances/:id/scan/report.json | Get the scan report. |
Instances
| Method | Resource | Parameters | Description |
|---|---|---|---|
GET | /instances | List all Instances. | |
POST | /instances | Scan options (Hash) | Create a new Instance with the given scan options. |
GET | /instances/:id | Get progress info for Instance. | |
PUT | /instances/:id/scheduler | If a Scheduler has been set, put the Instance under its purview. | |
GET | /instances/:id/report.crf | Get a Cuboid report for the Instance. | |
GET | /instances/:id/report.json | Get a JSON report for the Instance. | |
PUT | /instances/:id/pause | Pause the Instance. | |
PUT | /instances/:id/resume | Resume the Instance. | |
DELETE | /instances/:id | Shutdown the Instance. |
Scan options
{
"url": "http://testhtml5.vulnweb.com",
"session": {
},
"audit": {
"parameter_values": true,
"mode": "moderate",
"exclude_vector_patterns": [
],
"include_vector_patterns": [
],
"link_templates": [
],
"links": true,
"forms": true,
"cookies": true,
"headers": true,
"ui_inputs": true,
"ui_forms": true
},
"scope": {
"directory_depth_limit": 10,
"auto_redundant_paths": 15,
"redundant_path_patterns": {
},
"dom_depth_limit": 4,
"dom_event_limit": 500,
"dom_event_inheritance_limit": 500,
"exclude_file_extensions": [
"gif",
"bmp",
"tif",
"tiff",
"jpg",
"jpeg",
"jpe",
"pjpeg",
"png",
"ico",
"psd",
"xcf",
"3dm",
"max",
"svg",
"eps",
"drw",
"ai",
"asf",
"rm",
"mpg",
"mpeg",
"mpe",
"3gp",
"3g2",
"avi",
"flv",
"mov",
"mp4",
"swf",
"vob",
"wmv",
"aif",
"mp3",
"mpa",
"ra",
"wav",
"wma",
"mid",
"m4a",
"ogg",
"flac",
"zip",
"zipx",
"tar",
"gz",
"7z",
"rar",
"bz2",
"bin",
"cue",
"dmg",
"iso",
"mdf",
"vcd",
"raw",
"exe",
"apk",
"app",
"jar",
"pkg",
"deb",
"rpm",
"msi",
"ttf",
"otf",
"woff",
"woff2",
"fon",
"fnt",
"css",
"js",
"pdf",
"docx",
"xlsx",
"pptx",
"odt",
"odp"
],
"exclude_path_patterns": [
],
"exclude_content_patterns": [
],
"include_path_patterns": [
],
"restrict_paths": [
],
"extend_paths": [
],
"url_rewrites": {
}
},
"http": {
"request_timeout": 20000,
"request_redirect_limit": 5,
"request_concurrency": 10,
"request_queue_size": 50,
"request_headers": {
},
"response_max_size": 500000,
"cookies": {
},
"authentication_type": "auto"
},
"device": {
"visible": false,
"width": 1600,
"height": 1200,
"user_agent": "Mozilla/5.0 (Gecko) SCNR::Engine/v1.0dev",
"pixel_ratio": 1.0,
"touch": false
},
"dom": {
"engine": "chrome",
"local_storage": {
},
"session_storage": {
},
"wait_for_elements": {
},
"pool_size": 10,
"job_timeout": 120,
"worker_time_to_live": 1000,
"wait_for_timers": false
},
"input": {
"values": {
}
},
"checks": [
],
"platforms": [
],
"plugins": {
},
"no_fingerprinting": false,
"authorized_by": null
}
Agent
| Method | Resource | Parameters | Description |
|---|---|---|---|
GET | /agent/url | Get the configured Agent URL. | |
PUT | /agent/url | URL (String) | Set the Agent URL. |
DELETE | /agent/url | Remove the Agent. |
Grid
| Method | Resource | Parameters | Description |
|---|---|---|---|
GET | /grid | Get Grid info. | |
GET | /grid/:agent | Get info of Grid member by URL. | |
DELETE | /grid/:agent | Unplug Agent from the Grid by URL. |
Scheduler
| Method | Resource | Parameters | Description |
|---|---|---|---|
GET | /scheduler | Get Scheduler info. | |
GET | /scheduler/url | Get Scheduler URL. | |
PUT | /scheduler/url | URL (String) | Set the Scheduler URL. |
DELETE | /scheduler/url | Remove the configured Scheduler. | |
GET | /scheduler/running | Get running Instances. | |
GET | /scheduler/completed | Get completed Instances. | |
GET | /scheduler/failed | Get failed Instances. | |
GET | /scheduler/size | Get queue size. | |
DELETE | /scheduler/ | Clear Scheduler queue. | |
POST | /scheduler/ | Scan options (Hash) | Push a scan to the Scheduler queue. |
GET | /scheduler/:instance | Get Instance info. | |
PUT | /scheduler/:instance/detach | Detach the given Instance from the Scheduler. | |
DELETE | /scheduler/:instance | Remove queued Instance job from the Scheduler queue. |
Examples
Starting the REST server
Start the server by issuing the following command:
bin/scnr_rest_server
Client
#!/usr/bin/env ruby
require 'pp'
require_relative 'http-helpers'
# Create a new scanner Instance (process) and run a scan with the following options.
request :post, 'instances', {
# Scan this URL.
url: 'http://testhtml5.vulnweb.com',
# Audit all element types.
audit: {
elements: [:links, :forms, :cookies, :headers, :jsons, :xmls, :ui_inputs, :ui_forms]
},
# Load all active checks.
checks: 'active/*'
}
# The ID is used to represent that instance and allow us to manage it from here on out.
instance_id = response_data['id']
while sleep( 1 )
request :get, "instances/#{instance_id}/scan/progress", {
# Get the hash-map representation of native objects, like issues.
as_hash: true,
# Include these types of objects only.
with: [:issues, :sitemap, :errors]
}
# Print out instance progress.
pp response_data
# Continue looping while instance status is 'busy'.
request :get, "instances/#{instance_id}"
break if !response_data['busy']
end
puts '*' * 88
# Get the scan report.
request :get, "instances/#{instance_id}/scan/report.json"
# Print out the report.
pp response_data
# Shutdown the Instance.
request :delete, "instances/#{instance_id}"
HTTP helpers
This client example included some helpers for the HTTP requests:
require 'json'
require 'tmpdir'
require 'typhoeus'
def response
if @last_response.headers['Content-Type'].include? 'json'
data = JSON.load( @last_response.body )
else
data = @last_response.body
end
{
code: @last_response.code,
data: data
}
end
def response_data
response[:data]
end
def request( method, resource = nil, parameters = nil )
options = {}
if parameters
if method == :get
options[:params] = parameters
else
options[:body] = parameters.to_json
end
end
options[:cookiejar] = "#{Dir.tmpdir}/cookiejar.txt"
options[:cookiefile] = options[:cookiejar]
@last_response = Typhoeus.send(
method,
"http://127.0.0.1:7331/#{resource}",
options
)
end
Web UI
The WebUI allows you to easily run, manage and schedule scans and their results via an intuitive web interface.
Boot-up
To boot the Pro interface please run:
bin/scnr_pro
After boot-up, you can visit the interface via your browser of choice.
Features
- Live scan progress.
- Scan coverage display.
- Parallel scans.
- Recurring scans – incremental scanning with automated issue reviews:
- Fixed issues – Issues that don’t appear in subsequent scans.
- Regressions – Fixed issues that re-appeared in subsequent scans.
- Scheduled scans.
- Simple frequency configuration.
- Cronline frequency configuration support.
- Identification of conflicting future scans in calendar.
- Server/scanner/network health display.
- Powerful yet intuitive filtering.
- Website role management.
- Form login.
- Script login.
- Device emulation.
- Scan profiles.
- Extensive scan log.
Screenshots
Sites
List
Settings
Scans
New
Summary
Issues
Issue
Coverage
Health
Events
User roles
List
New
Profiles
List
Show
Devices
List
Settings
Run air-gapped
In order to run Codename SCNR in an air-gapped environment you need to:
- Place the license file at:
~/.scnr/license.key- Either by copying it over from a previous activation on an Internet-enabled machine, or;
- by activating on-line.
- Run an instance of
bin/scnr_check_serverin your local network – Provides functionality needed for SSRF types of checks.- Set the
SCNR_CHECK_SERVERenvironment variable to the check server URL – ex.http://10.1.1.1:9292.
- Set the
Generate reports
Pro
You can export scan results in several formats from the “Export” tab of an aborted or completed revision scan.
CLI
There are 2 reference report format types that you may encounter when using SCNR:
*.crf– Cuboid report file.*.ser–SCNR::Enginereport.
Both of these files can be handled by the CLI scnr_reporter utility in order
to convert them to a multitude of formats or print the results to STDOUT.
For example, to generate an HTML report:
bin/scnr_reporter --report=html:outfile=my_report.html.zip /home/user/.scnr/reports/report.ser
Or, to just print the report to STDOUT:
bin/scnr_reporter --report=stdout /home/user/.scnr/reports/report.ser
At the time of writing, bin/scnr_reporter --reporters-list yields:
[~] Available reports:
[*] ap:
--------------------
Name: AP
Description:
Awesome prints a scan report hash.
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.1.1
[*] html:
--------------------
Name: HTML
Description:
Exports the audit results as a compressed HTML report.
Options:
[~] outfile - Where to save the report.
[~] Type: string
[~] Default: 2022-01-24 23_17_13 +0200.html.zip
[~] Required?: false
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.4.4
[*] json:
--------------------
Name: JSON
Description:
Exports the audit results as a JSON (.json) file.
Options:
[~] outfile - Where to save the report.
[~] Type: string
[~] Default: 2022-01-24 23_17_13 +0200.json
[~] Required?: false
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.1.3
[*] marshal:
--------------------
Name: Marshal
Description:
Exports the audit results as a Marshal (.marshal) file.
Options:
[~] outfile - Where to save the report.
[~] Type: string
[~] Default: 2022-01-24 23_17_13 +0200.marshal
[~] Required?: false
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.1.1
[*] stdout:
--------------------
Name: Stdout
Description:
Prints the results to standard output.
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.3.3
[*] txt:
--------------------
Name: Text
Description:
Exports the audit results as a text (.txt) file.
Options:
[~] outfile - Where to save the report.
[~] Type: string
[~] Default: 2022-01-24 23_17_13 +0200.txt
[~] Required?: false
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.2.1
[*] xml:
--------------------
Name: XML
Description:
Exports the audit results as an XML (.xml) file.
Options:
[~] outfile - Where to save the report.
[~] Type: string
[~] Default: 2022-01-24 23_17_13 +0200.xml
[~] Required?: false
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.3.7
[*] yaml:
--------------------
Name: YAML
Description:
Exports the audit results as a YAML (.yaml) file.
Options:
[~] outfile - Where to save the report.
[~] Type: string
[~] Default: 2022-01-24 23_17_13 +0200.yaml
[~] Required?: false
Author: Tasos "Zapotek" Laskos <[email protected]>
Version: 0.2
Optimize scans
Left to its own devices, Codename SCNR will try to optimize itself to match any given circumstance, but there are limitations to what it can do automatically.
If a scan is taking too long, chances are that there are ways to make it go much faster by taking a couple of minutes to configure the system to closer match your needs.
In addition to performance, the following options also affect resource usage so you can experiment with them to better match your available resources as well.
- Only enable security checks that concern you
- Tailor the audit process to the platforms of the web application
- Ensure server responsiveness
- Balance RAM consumption and performance
- Reduce RAM consumption by avoiding large resources
- Don’t follow redundant pages
- Adjust the amount of browser workers
- Pick the audit mode that suits you best
Only enable security checks that concern you
By default, SCNR will load all checks, which may not be what you want.
If you are interested in high severity vulnerabilities or don’t care for things like discovery of common files and directories, and the like, you should disable the superfluous checks.
You can enable only active checks with:
--checks=active/*
Or skip the inefficient passive ones with:
--checks=*,-common_*,-backup_*,-backdoors
Tailor the audit process to the platforms of the web application
By default, the system will fingerprint the web application in order to deduce what platforms power it, thus enabling it to only send applicable payloads (instead of everything) which results in less server stress and bandwidth usage.
However, it is a good idea to explicitly set the platforms, if you know them, so as to play it safe and get the best possible results – especially since database platforms can’t be fingerprinted prior to the audit and their payloads are a large part of the overall scan.
You can specify platforms with:
--platforms=linux,mysql,php,apache
Ensure server responsiveness
By default, SCNR will monitor the response times of the server and throttle itself down if it detects that the server is getting stressed. This happens in order to keep the server alive and responsive and maintain a stable connection to it.
However, there are times with weak servers when they die before SCNR gets a chance to adjust itself.
You can bring up the scan statistics on the CLI screen by hitting Enter, in
which case you’ll see something like:
[~] Currently auditing http://testhtml5.vulnweb.com/ajax/popular?offset=0
[~] Burst response time sum 6.861 seconds
[~] Burst response count 29
[~] Burst average response time 1.759 seconds
[~] Burst average 0 requests/second
[~] Original max concurrency 10
[~] Throttled max concurrency 2
We can see that the server is having a hard time from the following values:
- Burst average: 3 requests/second
- Burst average response time 1.759
- Burst average: 0 requests/second
- Throttled max concurrency: 2
The response times were so high (1.75 seconds) that SCNR had to throttle its HTTP request concurrency from 10 requests to 2 requests, which would result in a drastically increased scan time.
You can lower the default HTTP concurrency and try again to make sure that the server at no point gets a stressful load:
--http-request-concurrency=5
Balance RAM consumption and performance
Most excessive RAM consumption issues are caused by large (or a lot of) HTTP requests, which need to be temporarily stored in memory in order for them to later be scheduled in a way that achieves optimal network concurrency.
To cut this short, having a lot of HTTP requests in the queue allows SCNR to be better at performing a lot of them at the same time, and thus makes better use of your available bandwidth. So, a large queue means better network performance.
However, a large queue can lead to some serious RAM consumption, depending on the website and type of audit and a lot of other factors.
As a compromise between preventing RAM consumption issues but still getting
decent performance, the default queue size is set to 50.
You can adjust this number to better suit your needs depending on the situation.
You can adjust the HTTP request queue size via the --http-request-queue-size option.
Reduce RAM consumption by avoiding large resources
SCNR performs a large number of analysis operations on each web page. This is usually not a problem, except for when dealing with web pages of large sizes.
If you are in a RAM constrained environment, you can configure SCNR to not download and analyze pages which exceed a certain size limit – by default, that limit is 500KB.
You can adjust the maximm allows size of HTTP response via the --http-response-max-size option.
Don’t follow redundant pages
A lot of websites have redundant pages like galleries, calendars, directory listings etc. which are basically the same page with the same inputs but just presenting different data.
Auditing the first (or first few) of such pages is often enough and trying to follow and audit them all can sometimes result in an infinite crawl, as can be the case with calendars.
SCNR provides 2 features to help deal with that:
- Redundancy filters: Specify
patternandcounterpairs, pages matching thepatternwill be followed the amount of times specified by thecounter.--scope-redundant-path-pattern
- Auto-redundant: Follow URLs with the same combinations of query parameters a
limited amount of times.
--scope-auto-redundant– Default is10.
Adjust the amount of browser workers
SCNR uses real browsers to support technologies such as HTML5, AJAX and DOM manipulation and perform deep analysis of client-side code.
Even though browser operations are performed in parallel using a pool of workers, the default pool size is modest and operations can be time consuming.
By increasing the amount of workers in the pool, scan durations can be dramatically shortened, especially when scanning web applications that make heavy use of client-side technologies.
Finding the optimal pool size depends on the resources of your machine (especially the amount of CPU cores) and will probably require some experimentation; on average, 1-2 browsers for each logical CPU core serves as a good starting point.
However, do keep in mind that more workers may lead to higher RAM consumption as they will also accelerate workload generation.
You can set this option via --dom-pool-size.
The default is calculated based on the amount of available CPU cores your system has.
Pick the audit mode that suits you best
This is a Time vs Thoroughness balancing option.
--audit-mode:
quick– For a quick scan, complex or rare payloads will be omitted.moderate(default) – Balanced payloads.super– All payloads, more DOM probing, disabled attack optimization heuristics.
Maintain a valid session
Codename SCNR supports automated logout detection and re-login, as well as improved login procedures.
login_form plugin
The login_form plugin expects for following options:
url– The URL containing the login form;parameters– A URL-query-like string of form parameters;check– A pattern to be matched against the response body after requesting the supplied URL in order to verify a successful login.
After a successful login, the plugin will configure the system-wide session manager and let it know of the procedure it needs to follow in order to be able to login automatically, in case it gets logged out during the scan or the session expires.
Hint: If the response of the form submission doesn’t contain the check,
you can set a different check URL via the global --session-check-url option,
this will also require that a --session-check-pattern be set as well (it can
be the same as the autologin check option).
Limitations
This plugin operates a browser just like a regular user would and is thus limited to the same extent.
For example, if the login form is by default hidden and requires a sequence of UI interactions in order to become visible, this plugin will not be able to submit it.
For more complex sequences please use the login_script plugin.
Example
bin/scnr http://testfire.net --plugin=login_form:url=http://testfire.net/bank/login.aspx,parameters="uid=jsmith&passw=Demo1234",check="Sign Off|MY ACCOUNT" --scope-exclude-pattern=logout
The login form found in http://testfire.net/bank/login.aspx which contains the
uid and passw inputs will be updated with the given values and submitted.
After that, the response will be matched against the check pattern – which will
also be used for the duration of the scan to check whether or not the session is still valid.
(Since the “Sign Off” and “MY ACCOUNT” strings only appear when the user is logged-in, they are a reliable way to check the validity of the session.)
Lastly, we exclude (--scope-exclude-pattern) the logout link from the audit in order to avoid getting logged out.
login_script plugin
The login_script plugin can be used to specify custom login procedures, as simple
Ruby or JS scripts, to be executed prior to the scan and each time a logout is detected.
The script will be run under the context of a plugin, which means that it will have access to all system components, allowing you to login in the most optimal way – be that via a real browser, via HTTP requests, by loading an external cookie-jar file and many more.
With browser
If a browser is available, it will be exposed to the script
via the browser variable. Otherwise, that variable will have a value of nil.
If you require access to Selenium, browser.wd will provide you access to the appropriate WebDriver.
browser.goto 'http://testfire.net/bank/login.aspx'
form = browser.form( id: 'login' )
form.text_field( name: 'uid' ).set 'jsmith'
form.text_field( name: 'passw' ).set 'Demo1234'
form.submit
# You can also configure the session check from the script, dynamically,
# if you don't want to set static options via the user interface.
SCNR::Engine::Options.session.check_url = browser.url
SCNR::Engine::Options.session.check_pattern = /Sign Off|MY ACCOUNT/
With HTTP Client
If a real browser environment is not required for the login operation, then using the system-wide HTTP interface is preferable, as it will be much faster and consume much less resources.
response = http.post( 'http://testfire.net/bank/login.aspx',
parameters: {
'uid' => 'jsmith',
'passw' => 'Demo1234'
},
mode: :sync,
update_cookies: true
)
SCNR::Engine::Options.session.check_url = to_absolute( response.headers.location, response.url )
SCNR::Engine::Options.session.check_pattern = /Sign Off|MY ACCOUNT/
From cookie-jar
If an external process is used to manage sessions, you can keep SCNR in sync by loading cookies from a shared Netscape-style cookie-jar file.
http.cookie_jar.load 'cookies.txt'
Advanced session check configuration
In addition to just setting the check_url and check_pattern options, you can
also set arbitrary HTTP request options for the login check, to cover cases where
extra tokens or a method other than GET must be used.
framework.session.check_options = {
# :get, :post, :put, :delete
method: :post,
# URL query parameters.
parameters: {
'param1' => 'value'
},
# Request body parameters -- can also be a String instead of Hash.
body: {
'body_param1' => 'value'
},
cookies: {
'custom_cookie' => 'value'
},
headers: {
'X-Custom-Header' => 'value'
}
}
Proxy plugin
The proxy plugin can be used to train the system by inspecting the traffic
exchanged between the browser and the web application. From that traffic, it can
extract input vectors like links, forms and cookies from both sides – i.e. from
server responses as well as browser requests.
Since the proxy can inspect all this traffic, it can be instructed to record a login sequence and then deduce the login form and the values with which it was filled.
Like the form_login plugin, the proxy plugin will configure the system accordingly.
Example
bin/scnr http://testfire.net --plugin=proxy --scope-exclude-pattern=logout
You then need to configure your browser to use this proxy when connecting to the webapp, press the record button just before logging in and the stop button after.
You’ll then be presented with a simple wizard which will guide you through configuring a login check and verifying that the deduced login sequence works properly.
Lastly, we exclude (--scope-exclude-pattern=logout) the logout link from the audit in order to avoid getting logged out.
Cookie-jar
If the aforementioned techniques don’t work for you, you can pass a cookie-jar and manually configure the login-check using the following options:
--http-cookie-jar--session-check-url--session-check-pattern
This way SCNR will still be able to know if it gets logged out (which is helpful to several system components) but won’t be able to log-in automatically.
Of course, you should still exclude any path that can lead to the destruction of the session.
Scan services
At the moment the are no specialized service crawlers, however auditing web
services is possible by first training the system via its proxy plugin.
Capturing inputs
The best way to capture web service inputs is by running your service test-suite
and having its HTTP requests go through the proxy plugin.
Proxy plugin setup
You can setup the proxy like so:
bin/scnr http://target-url --scope-page-limit=0 --checks=*,-passive/* --plugin=proxy --audit-jsons --audit-xmls
The default proxy URL will be: http://localhost:8282
The --scope-page-limit=0 option tells the system to not do any sort of crawl
and only use what has been made visible via the proxy.
The --checks option tells the system to load all but irrelevant checks for
service scans – common files and directories and the like don’t really apply in this case.
The --audit-jsons --audit-xmls options restrict the scan to only JSON and XML inputs.
Test-suite setup
Test-suite configurations vary, however you can usually export the proxy setting as an environmental variable, prior to running your test-suite, like so:
export http_proxy=http://localhost:8282
If this global setting is ignored, you will need to explicitly configure your test-suite.
Exporting the input vectors
After running the test-suite, the system will have been trained with the input vectors of the web service. Thus, it would be a good idea to export that data, in order to avoid having to run the training scenarios prior to each scan.
The data can be retrieved with:
http_proxy=http://localhost:8282 curl http://scnr.engine.proxy/panel/vectors.yml -o vectors.yml
Starting the scan
In order for the scan to start you will need to shutdown the proxy:
http_proxy=http://localhost:8282 curl http://scnr.engine.proxy/shutdown
Re-using input vector data
Data exported via the proxy plugin can be imported via the vector_feed plugin, like so:
bin/scnr http://target-url --scope-page-limit=0 --checks=*,-passive/* --plugin=vector_feed:yaml_file=vectors.yml
Thus, you only have to run your test-suite scenarios once, for the initial training and then reuse the exported vector data for subsequent scans.
Debugging
You can debug the proxy manually via simple curl commands, like so:
http_proxy=http://localhost:8282 curl -H "Content-Type: application/json" -X POST -d '{ "input": "value" }' http://target-url/my-resource
Then, in SCNR’s terminal you’ll see something like:
[*] Proxy: Requesting http://target-url/my-resource
[~] Proxy: * 0 forms
[~] Proxy: * 0 links
[~] Proxy: * 0 cookies
[~] Proxy: * 1 JSON
[~] Proxy: * 0 XML
If you require further information, you can enable the --output-debug option;
acceptable verbosity values range from 1 to 3, 1 being the default.
Scale
SCNR can be configured into a Grid, in order to combine the resources of multiple nodes and thus perform large amounts of scans simultaneously or complete individual scans faster.
Its Grid can distribute workload horizontally and vertically and can also easily scale up and/or down.
In essence, Grids are created by connecting multiple Agents together, at which point a mesh network of Agents is formed. Doing so requires no configuration, other than specifying an already running Agent when booting up a new one.
This allows for creating a private Cloud of scanners, with minimal configuration, that can handle an indefinite amount of workload.
Prior to continuing, it would be best if you took a look at SCNR’s distributed architecture.
Strategies
Scaling strategies can be configured via the --strategy option of Agents,
like so:
bin/scnr_agent --strategy=horizonstal
bin/scnr_agent --strategy=vertical
Horizontal (default)
SCNR Instances will be provided by the least burdened Agent, i.e. the Agent with the least utilization of its slots.
This strategy helps to keep the overall Grid health good by spreading the workload across as many nodes as possible.
Vertical
SCNR Instances will be provided by the most burdened Agent, i.e. the Agent with the most utilization of its slots.
This strategy helps to keep the overall Grid size (and thus cost) low by utilizing as few Grid nodes as possible.
It will also let you know if you have over-provisioned as extra nodes will not be receiving any workload.
Creating a Grid
In one terminal run:
bin/scnr_agent
This is the initial Agent.
Scaling up
To scale up just boot more Agents and specify a peer.
So, in another terminal run:
bin/scnr_agent --port=7332 --peer=127.0.0.1:7331
Lastly, in yet another terminal run:
bin/scnr_agent --port=7333 --peer=127.0.0.1:7332
(It doesn’t matter who the peer is as long as it’s part of the Grid.)
Now we have a Grid of 3 Agents.
The point of course is to run each Agent on a different machine in real life, but this will do for now.
Scaling down
You can scale down by unplugging an Agent from its Grid using:
bin/scnr_agent_unplug 127.0.0.1:7332
Running Grid scans
To start a scan that will be load-balanced across the Grid, simply issue
a spawn request on any of the Grid members.
Like so:
bin/scnr_spawn --agent-url=127.0.0.1:7331 http://testhtml5.vulnweb.com
The above will run a scan with the default options against http://testhtml5.vulnweb.com, originating from whichever node is optimal at any given time.
If the Grid is out of slots you will see the following message:
[~] Agent is at maximum utilization, please try again later.
In which case you can keep retrying until a slot opens up.
Running multi-Instance scans
The above is useful when you have multiple scans to run and you want to run them at the same time; another cool feature of SCNR though is that it can parallelize individual scans across the Grid thus resulting in huge single-scan performance gains.
For example, this would be useful if you were to scan a site with tens of thousands, hundreds of thousands or even millions of pages.
Even better, doing so is as easy as:
bin/scnr_spawn --agent-url=127.0.0.1:7331 http://testhtml5.vulnweb.com --multi-instances=5
The --multi-instances=5 option will instruct SCNR to use 5 Instances to run this
particular scan, with the aforementioned Instances being of course load-balanced
across the Grid.
Copyright
Copyright 2024 Ecsypno.
All rights reserved.