python使用mitmproxy抓取浏览器请求的方法-创新互联
最近要写一款基于被动式的漏洞扫描器,因为被动式是将我们在浏览器浏览的时候所发出的请求进行捕获,然后交给扫描器进行处理,本来打算自己写这个代理的,但是因为考虑到需要抓取https,所以最后找到Mitmproxy这个程序。
成都创新互联公司成立于2013年,先为北辰等服务建站,北辰等地企业,进行企业商务咨询服务。为北辰企业网站制作PC+手机+微官网三网同步一站式服务解决您的所有建站问题。安装方法:
pip install mitmproxy
接下来通过一个案例程序来了解它的使用,下面是目录结构
sproxy
|utils
|__init__.py
|parser.py
|sproxy.py
sproxy.py代码
#coding=utf-8 from pprint import pprint from mitmproxy import flow, proxy, controller, options from mitmproxy.proxy.server import ProxyServer from utils.parser import ResponseParser # http static resource file extension static_ext = ['js', 'css', 'ico', 'jpg', 'png', 'gif', 'jpeg', 'bmp'] # media resource files type media_types = ['image', 'video', 'audio'] # url filter url_filter = ['baidu','360','qq.com'] static_files = [ 'text/css', 'image/jpeg', 'image/gif', 'image/png', ] class WYProxy(flow.FlowMaster): def __init__(self, opts, server, state): super(WYProxy, self).__init__(opts, server, state) def run(self): try: pprint("proxy started successfully...") flow.FlowMaster.run(self) except KeyboardInterrupt: pprint("Ctrl C - stopping proxy") self.shutdown() def get_extension(self, flow): if not flow.request.path_components: return '' else: end_path = flow.request.path_components[-1:][0] split_ext = end_path.split('.') if not split_ext or len(split_ext) == 1: return '' else: return split_ext[-1:][0][:32] def capture_pass(self, flow): # filter url url = flow.request.url for i in url_filter: if i in url: return True """if content_type is media_types or static_files, then pass captrue""" extension = self.get_extension(flow) if extension in static_ext: return True # can't catch the content_type content_type = flow.response.headers.get('Content-Type', '').split(';')[:1][0] if not content_type: return False if content_type in static_files: return True http_mime_type = content_type.split('/')[:1] if http_mime_type: return True if http_mime_type[0] in media_types else False else: return False @controller.handler def request(self, f): pass @controller.handler def response(self, f): try: if not self.capture_pass(f): parser = ResponseParser(f) result = parser.parser_data() if f.request.method == "GET": print result['url'] elif f.request.method == "POST": print result['request_content'] # POST提交的参数 except Exception as e: raise e @controller.handler def error(self, f): pass # print("error", f) @controller.handler def log(self, l): pass # print("log", l.msg) def start_server(proxy_port, proxy_mode): port = int(proxy_port) if proxy_port else 8090 mode = proxy_mode if proxy_mode else 'regular' if proxy_mode == 'http': mode = 'regular' opts = options.Options( listen_port=port, mode=mode, cadir="~/.mitmproxy/", ) config = proxy.ProxyConfig(opts) state = flow.State() server = ProxyServer(config) m = WYProxy(opts, server, state) m.run() if __name__ == '__main__': start_server("8090", "http")
另外有需要云服务器可以了解下创新互联scvps.cn,海内外云服务器15元起步,三天无理由+7*72小时售后在线,公司持有idc许可证,提供“云服务器、裸金属服务器、高防服务器、香港服务器、美国服务器、虚拟主机、免备案服务器”等云主机租用服务以及企业上云的综合解决方案,具有“安全稳定、简单易用、服务可用性高、性价比高”等特点与优势,专为企业上云打造定制,能够满足用户丰富、多元化的应用场景需求。
网站题目:python使用mitmproxy抓取浏览器请求的方法-创新互联
分享地址:http://scyanting.com/article/hjhdg.html