👇👇👇👇Course download Link blow 👇👇👇👇👇👇
Automate your pentests with Python: Module 1. Using operating system’s features.
After getting familiar with pre-course materials it’s time to move forward and start implementing our own tool. First module is related to OS features available in Python. So our tool will be a set of options — scanner, fuzzer, etc.. We will start with preparing the basis for each functionality plugin. We can call it a kind of eco-system where we can easily add any functionality we want as a plugin module.
🔰programming book :- https://mega.nz/folder/O6QDzCJR#EtoKmC2KcbK3gr0K_htpoQ
By the end of this first module we will have an implemented starter script for running our next activities, which will use logging and be able to use multithreading features to run several tasks simultaneosly.
1. Basic setup
2. Logging function
3. Command line arguments
4. Checking available plug-ins
5. Displaying help
🔰hacking course :- https://mega.nz/folder/2rZQTZwa#ge_RQ_sSX-M3w45y3W5G-w
First of all we need to create a structure for our project. I will implement in a way I think is better. Of course, you are free to modify it to fit more for you. I will create a single file called autopentest.py which is a starter script for all action. Every action will be implemented as a plugin, it will be in a separate file in separate folder /plugins.
यह भी पढे :- what is singnaling system number 7 (SS7)
Next thing to decide is what programming paradigm to follow, I prefer object oriented way at will implement plugins as a subclasses of base class PentestActionPlugin. So that in starter script we should get arguments from user — what plugin to start, and then take a set of options for that plugin. Every plugin should be able to return its help message, options list, description, etc…
Other critical thing for every such projects is logging. Logging should be available at the early steps to simplify development process. It will make debugging easy for sure.
Now let’s start coding.
First we should build out eco-system for move further. I will provide full source code — there are many comments which are quite self-describing, but we have to discuss several moments in each piece of code.
Ok, look at the logger.
'''
Created on 29 june 2015
'''
import os,errno,datetime
class Logger(object):
'''
This class describes a simple logger.
You are free to modify to reflect your needs.
This implementation will store logs in files.
'''
log_file = "autepentest.log" # default filename
log_folder = "." # default is current folder
def __init__(self, params=None):
'''
Constructor
'''
self.log_folder = params['log_folder']
pass
def get_full_path(self):
return self.log_folder + '/' + self.log_file
def log(self, message):
'''
Logs provided text message to file.
Used format:
[date time] message
'''
try:
fp = open(self.get_full_path(), "a")
except IOError as e:
if e.errno == errno.EACCES:
print '[Logger] Permission error: no access'
return
# Not a permission error.
raise
else:
with fp:
fp.write('['+datetime.datetime.now().isoformat()+'] ' + message + os.linesep)
print message
def self_check(self):
'''
Methods checks availability of logs folder and permissions
'''
# check folder first
if not os.path.isdir(self.log_folder):
# error. no log folder found
print 'Error: logging directory path is not a directory'
return False
if not os.access(self.log_folder, os.W_OK):
print 'Current user has no needed permissions on log folder...trying to set 0755'
print 'Running chmod("' + self.log_folder + '", 0755)'
os.chmod(self.log_folder, 0755)
print 'Done.'
#check log file
if not os.path.exists(self.get_full_path()):
#try to create file
print self.get_full_path() + ' not found... try to create it'
with open(self.get_full_path(), 'w') as f:
f.write('Log file initiated' + os.linesep)
# directory exists. check permissions
if(not os.access(self.get_full_path(), os.W_OK) or
not os.access(self.log_folder, os.W_OK)):
print 'Current user has no needed permissions...trying to set 0755'
print 'running chmod("' + self.log_folder + '", 0755)'
os.chmod(self.log_folder, 0755)
print 'done.'
os.chmod(self.get_full_path(), 0755)
print 'running chmod("' + self.get_full_path() + '", 0755)'
print 'done.'
# now recheck access
if(not os.access(self.get_full_path(), os.W_OK) or
not os.access(self.log_folder, os.W_OK)):
return False
else:
return True
return True
Here we use some operating system’s command. For example, os.chmod() is analog of unix chmod command and used to change permissions of file and directories.
With windows you can use the same command by the effect is slightly different. MS Windows system will ignore all bit in mode except bit for setting read-only flag.
To do exactly the same in Windows your code need be a little different. Here is an example of how to work with permissions from Python on win-machines. You need to use win32security module from pywin32.
All constants are in module ntsecuritycon.
import win32security
import ntsecuritycon as con
FILENAME = "whatever"
userx, domain, type = win32security.LookupAccountName ("", "User X")
usery, domain, type = win32security.LookupAccountName ("", "User Y")
sd = win32security.GetFileSecurity(FILENAME, win32security.DACL_SECURITY_INFORMATION)
dacl = sd.GetSecurityDescriptorDacl() # instead of dacl = win32security.ACL()
dacl.AddAccessAllowedAce(win32security.ACL_REVISION, con.FILE_GENERIC_READ | con.FILE_GENERIC_WRITE, userx)
dacl.AddAccessAllowedAce(win32security.ACL_REVISION, con.FILE_ALL_ACCESS, usery)
sd.SetSecurityDescriptorDacl(1, dacl, 0) # may not be necessary
win32security.SetFileSecurity(FILENAME, win32security.DACL_SECURITY_INFORMATION, sd)
To understand mapping of constants to access modes in Windows you can look at this description :
http://www.netid.washington.edu/documentation/domains/sddl.aspx
In Linux and OSX to get file permission you can you os.stat – it has attribute st_mode. You should mask out needed bits from it, for example, with contants already made for that task.
Like in this example:
import os
import stat
filepath = './folder'
st = os.stat(filepath)
print st.st_mode & stat.S_IRGRP
It checks if the filepath provided is readable to group.
Also look at the os.access description. It contains several contants you can use in your scripts like W_OK is for checking that we can write, R_OK for reading and X_OK for executing.
We used open() function to open file and set ‘w’ to set mode — writing in this case. ‘r’ will open file for reading only. That can ensure that you will not damage file while working with it, it will be opened in read-only mode.
This logger is not very smart for now, we definitely will upgrade it to make it write events to database for long term storing.
File is not a good choice if we want to query the events somehow.
Now its time to our starter script. It will deal with command line, parse arguments, provides help and other things.
It is quite simple now. Here it is.
#!/usr/local/bin/python2.7
# encoding: utf-8
'''
main.autopentest -- A tool for automating pentest routines autopentest
can scan hosts, make a simple fuzzing
'''
import sys
import os
from argparse import ArgumentParser
from argparse import RawDescriptionHelpFormatter
from starter import Starter
__all__ = []
__version__ = 0.1
__date__ = '2015-06-29'
__updated__ = '2015-06-29'
class CLIError(Exception):
'''Generic exception to raise and log different fatal errors.'''
def __init__(self, msg):
super(CLIError).__init__(type(self))
self.msg = "E: %s" % msg
def __str__(self):
return self.msg
def __unicode__(self):
return self.msg
def main(argv=None): # IGNORE:C0111
'''Command line options.'''
if argv is None:
argv = sys.argv
else:
sys.argv.extend(argv)
program_name = os.path.basename(sys.argv[0])
program_shortdesc = __import__('__main__').__doc__.split("\n")[1]
program_license = '''%s
Created by icewind on %s.
Copyright 2015 Vladimir Korennoy. All rights reserved.
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
''' % (program_shortdesc, str(__date__))
try:
# Setup argument parser
parser = ArgumentParser(description=program_license, formatter_class=RawDescriptionHelpFormatter)
parser.add_argument("-p", "--plugins_folder", dest="plugins_folder", help="plugins folder [default is ./plugins]")
parser.add_argument("-l", "--logs_folder", dest="logs_folder", help="folder for logs storing [default is ./logs]")
parser.add_argument("-v", "--verbose", dest="verbose", action="count", help="set verbosity level [default: %(default)s] 1 - only log 2-file and console")
parser.add_argument(dest="plugin_name", help="name of plugin to run")
parser.add_argument(dest="options", help="options for plugin", nargs="+", metavar="option")
# Process arguments
args = parser.parse_args()
verbose = args.verbose
options = dict( [arg.split('=') for arg in args.options] )
if verbose > 0:
print("Verbose mode on")
# Check logging folder
if(not starter.check_log_folder()):
print "Something wrong with logs folder. Please check it again"
return 0
else:
print 'Logger initiated successfully'
# Check plugins folder
if(not starter.check_plugins_folder()):
print "Something wrong with plugins folder. Please check it again"
return 0
else:
print 'Plugins directory is OK'
# configure plugin starter
starter = Starter({"log":args.logs_folder, "plugin":args.plugins_folder,
"plugin_name":args.plugin_name, "plugin_options":options})
# now run our plugin starter
starter.run()
return 0
except KeyboardInterrupt:
### handle keyboard interrupt ###
return 0
except Exception, e:
raise(e)
indent = len(program_name) * " "
sys.stderr.write(program_name + ": " + repr(e) + "\n")
sys.stderr.write(indent + " for help use --help")
return 2
if __name__ == "__main__":
sys.exit(main())
You can see there that we registered several command line arguments.
They are plugins_folder — to know where to look for plugins, verbosity and plugin name to run with its options.
And a Starter class which I will provide below. By now, our system will be able to run one plugin at a time. In next modules we will upgrade it to run several at one time. So, in short words, this piece of code handles the start of process and command line arguments. Thank to that we do not need to deal with it in our underlying scripts.
It takes options and plugin name, and including plugin in plugin_folder, then we use the Starter to run the plugin and give it options.
from logger import Logger
import os, inspect,sys, traceback
from plugin import Plugin
class Starter(object):
'''
Starter will check the state of filesystem.
Do we have plugins folder? do we have logs folder?
And permissions on a log folder to ensure we can write to it.
It will handle switching between plugins and other things.
'''
default_log_folder = "/Users/icewind/python_ws_practice/module2/autopentest/logs"
default_plugin_folder = "/Users/icewind/python_ws_practice/module2/autopentest/plugins"
log_folder = None
plugins_folder = None
plugin_options = None
loggerInstance = None
def __init__(self, params=None):
self.log_folder = params["log"] if params["log"] is not None else self.default_log_folder
self.plugins_folder = params["plugin"] if params["plugin"] is not None else self.default_plugin_folder
self.plugin_name = params["plugin_name"]
self.plugin_options = params["plugin_options"]
self.loggerInstance = Logger({'log_folder': self.log_folder})
def check_log_folder(self):
'''
Check if log folder exists and we have permissions to write files
'''
# this functionality is implemented as a self check in logger
return self.loggerInstance.self_check()
def check_plugins_folder(self):
'''
Check if plugins folder exists and we have permissions to write files
'''
if not os.path.isdir(self.plugins_folder):
# error. no log folder found
self.loggerInstance.log("Plugins directory " + self.plugins_folder + " not found")
return False
self.loggerInstance.log('Plugin directory is OK.. now checking access')
# directory exists. check permissions
if(not os.access(self.log_folder, os.R_OK)):
self.loggerInstance.log('Plugin directory is not writable')
return False
self.loggerInstance.log('Plugin directory is writable... OK')
return True
def load_plugin_class(self, module_name, class_name):
'''
Methods loads plugin class by name
'''
try:
module_ = __import__(module_name)
try:
class_ = getattr(module_, class_name)
return class_
except AttributeError:
self.loggerInstance.log('Class '+ class_name + ' does not exist')
return None
except ImportError:
self.loggerInstance.log('Module '+ module_name + ' does not exist')
traceback.print_exc(file=sys.stdout)
return None
def run(self):
'''
Methods runs a plugin.
Also gives options to it before running job
'''
plugin_class = self.load_plugin_class(self.plugin_name, self.plugin_name)
pluginInstance = plugin_class();
print pluginInstance.run(self.plugin_options)
So this is our main class which is responsible for running actual actions. It sets options for plugin, finds needed plugin and then runs it. The actual part of code that loads the plugin is this:
try:
module_ = __import__(module_name)
try:
class_ = getattr(module_, class_name)
#print class_
return class_
except AttributeError:
self.loggerInstance.log('Class '+ class_name + ' does not exist')
return None
except ImportError:
self.loggerInstance.log('Module '+ module_name + ' does not exist')
traceback.print_exc(file=sys.stdout)
return None
We got the name of plugin from command line arguments and assume that according file is in plugin folder which is set too. Next, we importing that to our space, getting class definition from plugin module using getattr() function.
The one thing here is that we decided that the class name of plugin should be equal to module name. But it is the only restriction we made. Our plugin classes are free to use any additional classes they need in their code. So, method load_plugin_class() method imports needed class. Next we create an instance of that class and call method run() on it.
pluginInstance = plugin_class();
print pluginInstance.run(self.plugin_options)
By now you may notice that we do not process the results. We just print what our plugin returns. Do not worry, later we implement it. At this moment we simply do not know what the result would be that’s why we can’t understand what to do with it.
It’s time to look at our plugin class, it will be base for all plugins we will write.
'''
Created on 29 june 2015
'''
from abc import ABCMeta, abstractmethod
from _pyio import __metaclass__
class Plugin(object):
'''
Base class for all plugins.
Every plugin should extend this class
to ensure correct behavior in infrastructure
'''
__metaclass__ = ABCMeta
logger = None
def __init__(self, params):
'''
Constructor
'''
@abstractmethod
def help(self):
'''
Displays help message for plugin.
Method should be overwritten in subclasses
'''
pass
@abstractmethod
def run(self, arguments):
'''
Main plugin method.
Runs the job and returns results.
'''
pass
To use several plugins in the same way they need to use same interface. For ensuring that plugins are all use same interface we will «abc» module which is included in python bundle starting from version 2.6. and is highly useful in such cases. ABC here stands for Abstract Base Classes.
The main class in this module is ABCMeta. With this module you can also define abstract properties along with abstract classes and methods.
Using this we can check inheritance with assert issubclass() function. You will see that our plugin will be a subclass of AbstractPlugin.
We used @abstractmethod decorator from that module. Using ABCMeta is simple and we get a good-readable code after it.
As you see, we will require that all further plugins should have methods run() and help().
The help() method is used to provide help description from plugin to environment since every plugin will have its own logic and its own help description.
As an alternative you can use Zope project features (zope.interfaces). Zope implementation is focused on component architecture and became more a set of independent components. It uses a concept of interfaces and adapters, based on these interfaces.
Look at this example, it is a bit similar approach that uses ABCMeta:
import zope.interface
class IVehicle(zope.interface.Interface):
"""Any moving thing"""
speed = zope.interface.Attribute("""Movement speed""")
def move():
"""Make a single step"""
class Car(object):
zope.interface.implements(IVehicle)
def __init__:
self.speed = 1
self.location = 1
def move(self):
self.location = self.speed*1
print "moved!"
assert IVehicle.implementedBy(Car)
assert IVehicle.providedBy(Car())
Here we declare which attributes and methods the object should have. And the class implements an interface, while the object provides. This is where the different is.
«Implementation» of the interface means that produced entity will have concrete features, while «providing» interface point to concrete features of current entity.
You need to understand that IVehicle declaration is a fiction, no real checks will be done. It is more like an agreement.
class IVehicle(zope.interface.Interface):
"""Any moving thing"""
speed = zope.interface.Attribute("""Movement speed""")
def move():
"""Make a single step"""
class Car(object):
zope.interface.implements(IVehicle)
assert IVehicle.implementedBy(Car)
assert IVehicle.providedBy(Car())
In simple examples interfaces do not help, but increase complexity. Zope architecture inclues another important term — adapters. In short words, adapter is a template which corrects one class for using elsewhere where different set of methods and attributes is needed.
After example I will provide below I think it should be clear.
Assume we have two classes, let they be Guest and Desk. Let’s try to define an interfaces for them and a class which will implement Guest interface.
import zope.interface
from zope.interface import implements
from zope.component import adapts, getGlobalSiteManager
class IDesk(zope.interface.Interface):
def register():
"Register a person"
class IGuest(zope.interface.Interface):
name = zope.interface.Attribute("""Person`s name""")
class Guest(object):
implements(IGuest)
def __init__(self, name):
self.name=name
Adapter should consider anonymous guest by registering him in name list:
class GuestToDeskAdapter(object):
adapts(IGuest)
implements(IDesk)
def __init__(self, guest):
self.guest=guest
def register(self):
guest_name_db.append(self.guest.name)
There is a register which holds a list of adapters and interfaces. Using it we can get adapter via giving object being adapted as an argument to it. If adapter is not registered second interface argument will be returned.
guest = Guest("NoOne")
adapter = IDesk(guest, alternate=None)
print adapter
>>>>None found
gsm = getGlobalSiteManager()
gsm.registerAdapter(GuestToDeskAdapter)
adapter = IDesk(guest, alternate="None found")
print adapter
>>>>__main__.GuestToDeskAdapter object at 0xb7beb64c>
Such approach is useful when you need to break code into components and link them together. One of the most successful example of using suck technique is Twisted framework — its architechture mainly relies on zope.interfaces
Now we can write plugins and use them. Here is the basic implementation I wrote to check that all works fine:
'''
Created on 29 june 2015
BasicPlugin.py
'''
from plugin import Plugin
from abc import abstractmethod
import datetime
class BasicPlugin(Plugin):
'''
Example plugin for demonstrating and debugging
features. It can return current datetime in 2 formats
ISO and using built-in python str() function
'''
logger = None
def __init__(self, params=None):
'''
Constructor.
Here we ignore params cause we will provide
them a different way in run() method to be able to
run plugin with different ones each time.
Plugin do not store its options. Its per execution
'''
pass
def help(self):
return '''
This is a help message for BasicPlugin. BP outputs current date.
Options:
use_iso = True/False Set the ISO or str() output format
Usage:
$ python autopentest.py BasicPlugin use_iso=True
'''
def run(self, arguments):
'''
This is a test plugin.
So its goal is to output current date and that is all
But it has one option - use_iso which indicates date output format
'''
use_iso = arguments['use_iso']
current_dt = datetime.datetime.now()
if(use_iso):
return current_dt.isoformat()
else:
return str(current_dt)
Now we can test it. Run it and you will see this:
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest/main [17:03:25]
$ python autopentest.py
usage: autopentest.py [-h] [-p PLUGINS_FOLDER] [-l LOGS_FOLDER] [-v]
plugin_name option [option ...]
autopentest.py: error: too few arguments
All works as expected, we didn't provide any arguments and received a help message with command line arguments format needed.
Now we can provide needed arguments:
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [15:39:57]
$ python main/autopentest.py -p /Users/icewind/python_ws_practice/module2/autopentest/plugins -l /Users/icewind/python_ws_practice/module2/autopentest/logs BasicPlugin use_iso=True
plugin name:BasicPlugin
/Users/icewind/python_ws_practice/module2/autopentest/logs
Logger initiated successfully
[2015-07-15T16:01:55.368368] Plugin directory is OK.. now checking access
[2015-07-15T16:01:55.368834] Plugin directory is writable... OK
Plugins directory is OK
[2015-07-15T16:01:55.381032] loading BasicPlugin
[2015-07-15T16:01:55.464686] Loaded class from module
[2015-07-15T16:01:55.464903] Calling plugin contructor...
[2015-07-15T16:01:55.465350] Basic Plugin inited
[2015-07-15T16:01:55.465682] run method called
2015-07-15T16:01:55.465728
As a result we see datetime printed. As was expected.
This was really useless plugin, but we tested our infrastructure and can now move to more useful ones.
Typical sub-routine while doing pentests is to download something and looks at contents. For example, imaging bruteforcing web page login form — you have to search to certain things like ‘Login successful’ or ‘Welcome, admin’ in response text from web server. That is why it will be very handy plugin in future. Lets call it GetFilePlugin.
Next think we will implement is StringMatcher plugin. This plugin will take URL and pattern and return True if given text is on this page and False if not. But here we test another interesting thing, the feature of our eco-system — this plugin will use previous GetFilePlugin in its internal process. And we will see that we can use our plugins anywhere we want. We can now construct plugin of a higher level of abstraction or automation.
Let’s start with GetFilePlugin.
Here is the code:
#coding: utf-8
'''
Created on 15 july 2015 г.
'''
import urllib
from plugin import Plugin
from logger import Logger
class GetFilePlugin(Plugin):
'''
Plugin for downloading files.
Files stored in plugin resources folder by default
'''
logger = None
def __init__(self, params=None):
self.logger = params['logger']
def help(self):
return '''
Plugin expects arguments:
a) 'url' - URL of target(full with http://)
b) 'path' - where to save downloaded object (full path)
c) 'name' - name of downloaded file
'''
def run(self, arguments):
'''
Downloads file and save to given location.
'''
self.logger.log('Starting file download...')
self.objectName = arguments['name']
self.targetUrl = arguments['url']
self.savePath = arguments['path']
fullpath = self.savePath + "/" + self.objectName
try:
urllib.urlretrieve(self.targetUrl, fullpath)
self.logger.log("File saved as " + fullpath)
return fullpath
except Exception as e:
self.logger.log("Download failed with error:" + e.message)
return None
def dependencies(self):
'''
Returns all needed for plugin to work correctly.
Returns a dictionary with names keys.
Values are dictionary with path, minimum version
Example:
["nmap",
"metasploit"]
'''
return ["urllib"]
def resultDescription(self):
'''
Returns a description of the plugin results
'''
return "The result is full path to downloaded file"
Mention here we use new methods from abstract plugin — dependencies and resultDescription.
For downloading file I used
urllib.urlretrieve(self.targetUrl, fullpath)
from urllib
But there are three typical methods for downloading a file. Here they are:
#first
import urllib2
with open('name','wb') as f:
f.write(urllib2.urlopen(URL).read())
f.close()
print "Complete!"
#second
import requests
r = requests.get(URL)
with open("fname", "wb") as code:
code.write(r.content)
print "Complete!"
#third
import urllib
urllib.urlretrieve(URL, "fname")
print "Complete!"
The difference is in used libraries. However, third methods have one line of code instead of 2-3 in others. So I chose this.
By the way, to be cross platform, consider using os.pathsep instead of «/» in:
fullpath = self.savePath + "/" + self.objectName
Let’s run it.
Here is the console output we will see:
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [12:22:53]
$ python main/autopentest.py -p /Users/icewind/python_ws_practice/module2/autopentest/plugins -l ./logs GetFilePlugin url=http://google.com path=. name=testfile
plugin name:GetFilePlugin
./logs
Logger initiated successfully
[2015-07-16T12:22:56.718769] Plugin directory is OK.. now checking access
[2015-07-16T12:22:56.718921] Plugin directory is writable... OK
Plugins directory is OK
[2015-07-16T12:22:56.719627] loading GetFilePlugin
[2015-07-16T12:22:56.728483] Loaded class from module
[2015-07-16T12:22:56.728861] Calling plugin contructor...
[2015-07-16T12:22:56.729099] Starting file download...
[2015-07-16T12:22:56.729244] Fullpath: ./testfile
[2015-07-16T12:22:56.869716] File saved as ./testfile
./testfile
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [12:22:56]
$ l
total 64
drwxrwxrwx 11 icewind staff 374B 16 июл 12:22 .
drwxr-xr-x 4 icewind staff 136B 29 июн 12:44 ..
-rwxrwxrwx 1 icewind staff 365B 29 июн 12:44 .project
-rwxrwxrwx 1 icewind staff 465B 29 июн 12:44 .pydevproject
drwxrwxrwx 3 icewind staff 102B 29 июн 15:22 .settings
-rwxrwxrwx 1 icewind staff 0B 15 июл 14:27 __init__.py
drwxrwxrwx 3 icewind staff 102B 16 июл 11:30 logs
drwxrwxrwx 10 icewind staff 340B 16 июл 12:22 main
drwxrwxrwx 9 icewind staff 306B 16 июл 11:31 plugins
drwxrwxrwx 3 icewind staff 102B 14 июл 17:27 resources
-rw-r--r-- 1 icewind staff 21K 16 июл 12:22 testfile
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [12:22:57]
$
And now StringMatcherURL plugin. Give it url and pattern — a regular expression to find on page. If it is there — plugin return True. We will use it for testing the reaction to our pentest actions.
It is a very typical task which we will automate. It is very boring, for example, to do sql injection by hand and every time to look at the page to find any results. All such actions should be automated in ny opinion.
#coding: utf-8
'''
Created on 15 july 2015 г.
'''
import hashlib,re,os
from plugin import Plugin
from logger import Logger
from GetFilePlugin import GetFilePlugin
class StringMatcherURL(Plugin):
'''
This plugin matche a string on contents of given url.
Typically used to check presence of string on a page
'''
logger = None
def __init__(self, params=None):
self.logger = params['logger']
self.logger.plugin_name = 'StringMatcherURL'
def help(self):
return '''
Plugin expects arguments:
a) 'url' - URL of target(full with http://)
b) 'pattern' - a string to search for
'''
def run(self, arguments):
'''
Downloads page and searches for a given string
'''
self.logger.log('Starting StringMatcherURL plugin...')
self.pattern = arguments['pattern']
self.targetUrl = arguments['url']
# here we will use our previous plugin
getFilePlugin = GetFilePlugin({'logger':Logger({'plugin_name':'GetFilePlugin', 'log_folder':'./logs'})})
hdigest = hashlib.sha224(self.targetUrl + self.pattern).hexdigest()
filename = getFilePlugin.run({'url':self.targetUrl, 'path': '.', 'name': hdigest})
self.logger.log('Saving file:' + filename)
with open (filename, "r") as myfile:
data = myfile.read().replace('\n', '')
result = re.search(self.pattern, data)
#remove temp file
os.remove(filename)
#if result is None that there are no matches
if result is None:
self.logger.log('Result ready: Match NOT found!')
return False
else:
self.logger.log('Result ready: Match found!')
return True
def dependencies(self):
'''
Returns all needed for plugin to work correctly.
Returns a dictionary with names keys.
Values are dictionary with path, minimum version
Example:
["nmap",
"metasploit"]
'''
return ["GetFilePlugin"]
def resultDescription(self):
'''
Returns a description of the plugin results
'''
return "Returns true if match found on page, False otherwise"
I used hashlib module here which provides several hashing algorithms. This used to ensure that our files will not have the same names. If we start to use plugin intensively, there will be many temporary files saved, and we need to use unique names to differ them. So I used hash from given URL and pattern — it will identify the result.
Another thing is dependencies — we set GetFilePlugin as dependency because we use it to download the page from URL.
Now we see that we can use our plugins in other plugins. It is a very powerful ability. It means we can create plugins for different actions and then combine it to gain other things.
Let’s run and test it.
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [12:27:49]
$ python main/autopentest.py -p ./plugins -l ./logs StringMatcherURL url=http://google.com pattern=124
plugin name:StringMatcherURL
./logs
Logger initiated successfully
[2015-07-16T12:28:32.004077] Plugin directory is OK.. now checking access
[2015-07-16T12:28:32.004234] Plugin directory is writable... OK
Plugins directory is OK
[2015-07-16T12:28:32.004388] loading StringMatcherURL
[2015-07-16T12:28:32.062883] Loaded class from module
[2015-07-16T12:28:32.063019] Calling plugin contructor...
[2015-07-16T12:28:32.063179] Starting StringMatcherURL plugin...
[2015-07-16T12:28:32.063388] Starting file download...
[2015-07-16T12:28:32.063537] Fullpath: ./50e808dc3933be398d8c0351f388190360e00962a071b6e2ba495366
[2015-07-16T12:28:32.407793] File saved as ./50e808dc3933be398d8c0351f388190360e00962a071b6e2ba495366
[2015-07-16T12:28:32.408832] Saving file : ./50e808dc3933be398d8c0351f388190360e00962a071b6e2ba495366
[2015-07-16T12:28:32.409717] Result ready: Match NOT found!
False
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [12:28:32]
$ l
total 64
drwxrwxrwx 11 icewind staff 374B 16 июл 12:28 .
drwxr-xr-x 4 icewind staff 136B 29 июн 12:44 ..
-rwxrwxrwx 1 icewind staff 365B 29 июн 12:44 .project
-rwxrwxrwx 1 icewind staff 465B 29 июн 12:44 .pydevproject
drwxrwxrwx 3 icewind staff 102B 29 июн 15:22 .settings
-rwxrwxrwx 1 icewind staff 0B 15 июл 14:27 __init__.py
drwxrwxrwx 3 icewind staff 102B 16 июл 11:30 logs
drwxrwxrwx 10 icewind staff 340B 16 июл 12:22 main
drwxrwxrwx 9 icewind staff 306B 16 июл 12:28 plugins
drwxrwxrwx 3 icewind staff 102B 14 июл 17:27 resources
-rw-r--r-- 1 icewind staff 21K 16 июл 12:22 testfile
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [12:28:37]
$
Ok! It works great! We looked at page at google.com for a pattern ‘123’ and its not found. That is true. Second command here for checking that we haven’t leave temporary files there.
The last useful plugin for our start is some plugin which can store results. It will be used very often. Every time during pentest we will need to save something we found, or to save it in order to provide it to other plugin — that is a task for this plugin.
It will be a very simple task since we have all the infrastructure already, and already familiar with functions to write files. So it is a part of GetFilePlugin indeed, all we need to do is to modify it not to get data from URL but from input provided.
Here it is:
#coding: utf-8
'''
Created on 15 july 2015 г.
'''
import urllib
from plugin import Plugin
from logger import Logger
class FileSaver(Plugin):
'''
Plugin for saving data to files.
Files stored provided data into file.
It can rewrite or append it
'''
logger = None
def __init__(self, params=None):
self.logger = params['logger']
def help(self):
return '''
Plugin expects arguments:
a) 'data' - data to write
b) 'path' - path to file
c) 'append' - if True then append data, otherwise rewrite
'''
def run(self, arguments):
'''
Saves the given data to file
'''
self.logger.log('Starting FileSaver plugin.')
self.data = arguments['data']
self.savePath = arguments['path']
self.append = self.ParseBoolean(arguments['append'])
filemode = "a" if self.append is True else "w"
print filemode
try:
fp = open(self.savePath, filemode)
with fp:
fp.write(self.data)
self.logger.log("File saved as " + self.savePath)
return None
except Exception as e:
self.logger.log("FileSaver failed with error:" + e.message)
return None
def ParseBoolean (self, b):
if len(b) < 1:
raise ValueError ('Cannot parse empty string into boolean.')
b = b.lower()
print b
if b == 't' or b == 'y' or b == '1' or b == "true":
return True
if b == 'f' or b == 'n' or b == '0' or b == "false":
return False
raise ValueError ('Cannot parse string into boolean.')
def dependencies(self):
'''
Returns all needed for plugin to work correctly.
Returns a dictionary with names keys.
Values are dictionary with path, minimum version
Example:
["nmap",
"metasploit"]
'''
return [] # no dependencies for simple file writing
def resultDescription(self):
'''
Returns a description of the plugin results
'''
return "The result is None. File saved and no response expected"
And the corresponding output will be like this:
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [13:13:23]
$ python main/autopentest.py -p ./plugins -l ./logs FileSaver data="this is a test data" path=./savedFile append=False
plugin name:FileSaver
./logs
Logger initiated successfully
[2015-07-16T13:13:24.735892] Plugin directory is OK.. now checking access
[2015-07-16T13:13:24.736348] Plugin directory is writable... OK
Plugins directory is OK
[2015-07-16T13:13:24.749749] loading FileSaver
[2015-07-16T13:13:24.837914] Loaded class from module
[2015-07-16T13:13:24.838055] Calling plugin contructor...
[2015-07-16T13:13:24.838213] Starting FileSaver plugin.
[2015-07-16T13:13:24.881767] File saved as ./savedFile
None
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [13:13:24]
$ l
total 72
drwxrwxrwx 12 icewind staff 408B 16 июл 13:13 .
drwxr-xr-x 4 icewind staff 136B 29 июн 12:44 ..
-rwxrwxrwx 1 icewind staff 365B 29 июн 12:44 .project
-rwxrwxrwx 1 icewind staff 465B 29 июн 12:44 .pydevproject
drwxrwxrwx 3 icewind staff 102B 29 июн 15:22 .settings
-rwxrwxrwx 1 icewind staff 0B 15 июл 14:27 __init__.py
drwxrwxrwx 3 icewind staff 102B 16 июл 11:30 logs
drwxrwxrwx 10 icewind staff 340B 16 июл 12:22 main
drwxrwxrwx 11 icewind staff 374B 16 июл 13:13 plugins
drwxrwxrwx 3 icewind staff 102B 14 июл 17:27 resources
-rw-r--r-- 1 icewind staff 19B 16 июл 13:13 savedFile
-rw-r--r-- 1 icewind staff 21K 16 июл 12:22 testfile
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [13:13:26]
$ cat savedFile
this is a test data%
Our data is successfully saved to file. Now we will try to append some text to check that our flag works correctly:
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [13:35:25]
$ python main/autopentest.py -p ./plugins -l ./logs FileSaver data="33333333" path=./savedFile append=True
plugin name:FileSaver
./logs
Logger initiated successfully
[2015-07-16T13:35:36.197409] Plugin directory is OK.. now checking access
[2015-07-16T13:35:36.197673] Plugin directory is writable... OK
Plugins directory is OK
[2015-07-16T13:35:36.211578] loading FileSaver
[2015-07-16T13:35:36.291890] Loaded class from module
[2015-07-16T13:35:36.292060] Calling plugin contructor...
[2015-07-16T13:35:36.292250] Starting FileSaver plugin.
true
a
[2015-07-16T13:35:36.325813] File saved as ./savedFile
None
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [13:35:36]
$ cat savedFile
this is a test data33333333%
# icewind at IcewindDale in ~/python_ws_practice/module2/autopentest [13:35:40]
$
It’s great! Out plugin can append data. It is useful to constructing a complex report from many plugins in one file.
I think you mentioned that it is not very handy to set all the arguments every time. Plugin folder, log folder. That is totally true :) And we will fix that! It will be useful to create a configuration file with settings that do not change frequently, like these folders. So, it is not very hard to do.