• home
  • dradis framework guides

Import Plugins

An Import plugin is a special type of server plugin (see Plugins Overview for other types).

In this guide you will learn how to create and use new Import plugins.

1 Introduction

Import plugins let you search in remote sources of information and import back the results to the Dradis repository.

You can access the Import plugins through the Import note... tab in Dradis web interface:

We are going to go through the steps to create a new import plugin to search for vulnerabilities in the Open Source Vulnerability Database (OSVDB).

2 Define Plugin Goals

We are going to use OSVDB’s API to query the database, so we will need an API key that we can get from: https://osvdb.org/account/signup.

Our plugin will have two filters (i.e ways of querying the OSVDB database):

  • OSVDBIDLookup: used to lookup a specific OSVDB ID.
  • GeneralSearch: to run a query with a custom search string.

In this tutorial we will cover the first one. The full code of the plugin is available in the project’s git repository:

https://github.com/dradis/dradisframework/tree/master/vendor/plugins/osvdb_import

3 Generate the Plugin

The framework comes with a few handy plugin generators that will help you getting your plugin kickstarted.

While you develop your plugin, it will be easier to run Dradis in development mode, this means that a number of files will be auto-reloaded whenever you change them to make your life easier. You just need to initialize your development database with:


$ bundle exec thor dradis:reset

This should create a ./db/development.sqlite3 SQLite3 file.

Now we are ready to generate a new plugin and debug it in development mode. Go to the server folder and run:


$ bundle exec rails generate import_plugin osvdb

That will generate all the files we are going to need for the plugin.

All the generated code have been put in: ./vendor/plugins/osvdb_import/.

4 Create the Test Cases

To know where we are going, we start the plugin development by figuring out what test cases we are going to need to verify that our code does what it is meant to do.

We will create two unit tests:

  • One to verify that we have a valid API key.
  • Another to verify we are pulling the right information from the server.

Edit ./test/osvdb_import_test.rb:

  
def test_no_API_key
  OSVDBImport::CONF['API_key'] = OSVDBImport::BAD_API_KEY

  assert_raise RuntimeError do
    OSVDBImport::Filters::validate_API_key()
  end
end

So we will implement a validate_API_key() method that would ensure that the configured API key is valid.

As we will see in the next section, the test case code makes use of the OSVDBImport::CONF constant. This constant is generated by the import_plugin generator and will hold all the configuration settings of your plugin.

The test case can be extended to ensure that the different filters are making use of such function.

We are not providing the full code for the test cases in this tutorial for the sake of clarity, but the bottom line is: write your test cases before your code.

5 Plugin Configuration

If you check the ./lib/osvdb_import.rb you will see the following code:


  # OsvdbImport

  require 'osvdb_import/filters'
  require 'osvdb_import/meta'

  module FooImport
    class Configuration < Core::Configurator
      configure :namespace => 'osvdb_import'

      # setting :my_setting, :default => 'Something'
      # setting :another, :default => 'Something Else'
    end
  end

  # This includes the import plugin module in the dradis import plugin repository
  module Plugins
    module Import
      include OsvdbImport
    end
  end

The plugin generator created a plugin Configuration class and included this code to use the framework’s configuration storage facility. All the settings configured through the Configuration class are accessible via the Configuration Manager (https://localhost:3004/configurations).

The generator also included this plugin in the framework’s Import plugin collection (Plugins::Import).

We can customize our plugin settings:


  # [...]
  module OSVDBImport
    # Please register an account in the OSVDB site to get your API key. Steps:
    # 1. Create the account: http://osvdb.org/account/signup
    # 2. Find your key in http://osvdb.org/api
    class Configuration < Core::Configurator
      configure :namespace => 'osvdb'
      setting :api_key, :default => "<your_API_key>"
    end

    BAD_API_KEY = '<your_API_key>'
  end
  # [...]  

The only configuration setting our plugin needs is an OSVDB API key. The user needs to register on OSVDB’s site to get one and use the Configuration Manager to store it so the plugin can use it.

6 Implement the functionality

Import filters are defined in ./lib/osvdb_import/filters.rb. You can have a look at the filters created by the generator.

Each filters is defined inside a module under Osvdb::Filters with the following structure:


module OsvdbImport  
  module Filters

    module FilterName
      NAME = "Filter description, what the filter does."

      def self.run(params={})
        records = []
        # [...]
        # filter magic
        # [...]
        return records
      end
    end

    # other filters
    # [...]

Each filter module defines a NAME constant used by the Dradis interface to let the user know what the filter does.

Each filter module also defines a self.run() method that must return an array of records (the results). Each record is a Ruby hash that contains two fields: :title and :description. For instance:


  { :title => "Lynx Remote Overflow", :description => "lorem ipsum dolor sit amet" }

The filter task is to process the request made by the user (received through the params argument) and return a list of results in the appropriate format.

If you need to present an error to the user (for instance, if the OSVDB API key is invalid), you can return a record with the error description like this:


# [...]

# Ensure that we have a valid OSVDB API key
begin
  Filters::validate_API_key()
rescue Exception => e
  return [
    { :title => 'Error in OSVDB API key', :description => e.message}
  ]
end

# [...]

Each filter will retrieve the results in a different way depending on how these are available at the source of the information (in this case the OSVDB site).

Lets have a look at the implementation of the OSVDBIDLookup filter:


module OSVDBImport
  module Filters
    # [...] other filters
    
    # OSVDIDLookup: run a search looking for a specific OSVDID
    #
    # Sample Query for osvdb_id 1234: 
    #   http://osvdb.org/api/find_by_osvdb/<your_API_key>/1234
    module OSVDBIDLookup
      NAME = 'OSVDB ID Lookup'

      def self.run(params={})
        # Ensure that we have a valid OSVDB API key
        begin
          Filters::validate_API_key()
        rescue Exception => e
          return [
            { :title => 'Error in OSVDB API key', :description => e.message}
          ]
        end

        logger = params.fetch( :logger, Rails.logger )
        query = CGI::escape( params.fetch( :query, '1234') )

        logger.info{ "Running a OSVDB ID lookup on: #{query}" }
        results = OSVDB::IDLookup( :API_key => Configuration.api_key, :osvdb_id => query )

        return Filters::from_OSVDB_to_dradis(results)
      end
    end
    
    # [...] other filters
  end
end

In our example the first thing we do is to ensure that we have a valid API key with a call to Filters::validate_API_key

If we do, then we make a call to the OSVDB::IDLookup (more on this library later) to fetch the results.

Finally we use the Filters::from_OSVDB_to_dradis helper method to convert the results from the OSVDB format to the record format expected by the framework.

In order to avoid cluttering in the filter definition I placed all the OSVDB fetching code in a separate file, the OSVDB module (./osvdb_import/lib/osvdb.rb). It handles the HTTP connection and XML parsing of the response.

7 Add command line tasks

The last step in the process is to make your plugin’s functionality available through the command line.

This is done by virtue of Thor (simple ruby build program with capabilities similar to make/Rake).

This is a good idea also for debugging purposes. Rails plugins do not get reloaded unless you restart the server. So in order for a change in your code to become used by the framework, you need to restart Dradis (which is a slow process). Running your plugin through the console using the thor will make debugging easier.

The plugin generator defined a dummy thor task in ./tasks/thorfile.rb:


class DradisTasks < Thor
  class Import < Thor
    namespace "dradis:import"

#    desc "osvdb QUERY", "import from osvdb"
#    long_desc "This will appear if the user runs 'thor help dradis:import:osvdb'"
#    def osvdb(query)
#      require 'config/environment'
#
#    end

  end
end

We will replace it a slightly more complex (and useful) one:


class DradisTasks < Thor
  class Import < Thor
    class OSVDB < Thor
      namespace "dradis:import:osvdb"

      # [...]

      desc "lookup ID", "search the OSVDB for a specific ID"
      def lookup(id)
        require 'config/environment'

        results = OSVDBImport::Filters::OSVDBIDLookup.run(:query => id)

        puts "OSVDB Search\n============"
        puts "#{results.size} results"

        results.each do |record|
          puts "#{record[:title]}\n\t#{record[:description]}"
        end
      end

      # [...]

    end
  end
end  

You can add as many tasks as you want and use an Ruby code you need. Typically you will add one task per import Filter you have created.

In a nutshell this task invokes the OSVDBImport::Filters::OSVDBIDLookup filter with a :query that the user passes as a command line parameter. Afterwards, it prints the results.

Note that the final version of the plugin makes use of Thor’s namespaces. The art of Thor tasks is complex and we can’t cover much depth here. Dradis ships with tons of tasks (run bundle exec thor -T to see a list) and Thor’s site is another source of good information.

8 More Information

So that is about it. You are good to go, all the components are in place and the new plugin should be integrated with your Dradis server as shown below:

A screenshot of the Dradis web interface showing the OSVDB Import plugin in action

Import plugins as any Dradis server plugin are just standard Ruby on Rails plugins with a specific structure and purpose.

You can learn more about Rails plugins at the Ruby on Rails plugins guide.