Cisco Releases new Firepower/FTD 6.3 code!
Well, the release of Firepower 6.3 is now upon us! This release brings several long awaited features including multi-instance and FQDN Access Control rules. Let’s take a closer look at some of the highlights.
Licensing
There is a new Specific License Reservation available for approved customers. This allows the using the Firepower Management Center (FMC) on an air-gapped network. Previously, the only way to use the smart license feature was to have either Internet access or a satellite license server. Now a special license can be installed which is valid for a specific duration without the requirement to access the Cisco site or the Smart Software Satellite Server.
New/modified screens: System → Licenses → Specific Licenses
** This is detailed in it’s own blog: https://www.lammle.com/post/cisco-firepower-ftd-6-3-new-licensing-feature/
Interface Features
Another often requested feature has been hardware bypass modules for the 2100 series Firepower devices. With 6.3 you can now use hardware bypass network modules for your transparent inline sets. This provides an added safety net to allow traffic to pass if the device loses power. Bypass can also be enabled from the FMC web user interface under Device Management.
Access Control
There is now a configurable update interval for URL category and reputation data. If you are using the URL license to control the sites your users can access you may be interested in this change. To keep your URL data more up-to-date you can reduce this interval. As noted in the documentation, this is a tradeoff between accuracy and performance. A shorter update interval will keep your URL categorization data more current however it could result in slower web browsing for your users. You should test in your own environment to determine the best setting. Note that while the option to expire this data is present in 6.3 the default remains the same – cached URL data does not expire.
New/modified screens: System → Integration → Cisco CSI → Cached URLs Expire
Intelligent Application Bypass (IAB) is now enabled by default.
This is a fairly important under the hood setting in the Advanced tab of the Access Control policy. It may not be something you’ve enabled in the past but it’s worth looking at. The setting controls Snort’s behavior when there are large traffic flows which can tie up CPUs and cause high utilization and increased traffic latency. On systems such as the ASA with Firepower Services these large flows can even impact the other flows on the device. Intelligent Application Bypass is a highly configurable and automatic way to bypass these flows once certain thresholds are reached.
New Access Control policies created in version 6.3 will enable the IAB feature. Existing policies will not be changed. Keep in mind that this change in policy defaults can introduce unwanted deviations across your policies. If you create a new Access Control policy and assume it will have the same default settings as your existing policies you can wind up with inconsistent coverage in your environment. One feature that can help in this regard is policy hierarchy. Using a master Access Control policy to enforce settings on child policies can keep your environment in sync even when defaults change between versions.
Access Control and prefilter rules can now use fully qualified domain (FQDN) network objects.
You must configure DNS server groups and DNS platform settings so the system can resolve DNS names. This has been an often requested feature and brings FTD closer to parity with the ASA.
You can find the DNS Server group in Objects, and must also be configured in Platform settings for the FTD devices
High Availability and Scalability
Probably the biggest new feature in 6.3 is Multi-instance or, the ability to deploy multiple logical devices on a single security engine/module. This applies to the 4100 an 9300 devices only. This is not your father’s multi-context! Instead of processes sharing computing power like multi-context, Multi-instance allows you to allocate a number of CPUs to each instance.
The underlying technology uses Docker to run several virtual appliances on the same security module. Each instance looks like a separate FTD device to the FMC and each is managed separately. All processing is isolated to the instance so the CPU utilization of one instance cannot affect other instances on the same module. The number of instances available depends on the hardware model.
In this release you can setup high availability (HA) between instances on different chassis but clustering support will come in a later release. Also, Multi-instance currently only supports FTD. Hopefully we will see the ability to run both FTD and ASA instances on a single engine/module in a future release!
You also get this new Multi-instance capability without buying any additional licenses. Licensing is still done at the engine/module level so all the instances you create on the module use the same licenses already purchased.
Encryption
SSL hardware acceleration has now been extended to the 2100 series Firepower devices. Similar to their big brothers the 4100 and 9300, the 2100 can now use it’s on-board crypto chip to improve SSL decryption performance.
Note that upgrading to 6.3 automatically enables SSL hardware acceleration on eligible devices. If you are not using SSL decryption in your deployment, Cisco recommends disabling this setting on your devices that are not decrypting traffic.
The above note also applies to anyone who is using SSL decryption on a device that previously supported hardware acceleration. If you previously did not have this feature enabled, it will be automatically enabled upon upgrading to 6.3. This could be important as the hardware decryption chip does not support all of the cipher suites that are supported by software decryption. Make sure you test thoroughly to ensure you are not breaking some of your SSL sessions by enabling hardware acceleration.
Events, Logging and Analysis
One of the less touted but still interesting features is Contextual Cross-Launch. This makes external integrations with the FMC event views possible. There are dozens of cross-launch integration links included and you can even create your own custom links. Very handy for pivoting to other systems to gain additional context around a Firepower event.
Look under: Analysis → Advanced → Contextual Cross-Launch
Event logging via syslog has been improved.
There have been some format changes for syslog messages for connection, security intelligence and intrusion events. You may need to update your SIEM systems to ensure syslog messages are properly parsed.
The Access Control policy now has a Logging tab which consolidates several types of logging.
Access Control, Prefilter, SSL and Intrusion policy external syslog is now controlled by the settings on this tab. We are hoping for additional consolidations in the future bringing the ASA logs into the same central configuration scheme.
Administration and Troubleshooting
There have been a number of changes and improvements in the administration and troubleshooting areas. Some of these include the ability to set an access list for SNMP on devices. This is something classic Firepower has had for over a decade but is just finding its way into FTD. Also, you can now lock down the command line on the FMC by implementing a limited CLI and disabling the bash shell.
Additional login security options have been added for FMC users including tracking successful logins, limiting password reuse and disabling access temporarily for multiple login failures.
You can also now backup and restore FTD device configurations with a pull or pull option. You can also copy configurations between devices. This should speed up deployment of multiple devices and also help with device recovery.
Those are some highlights of the new Firepower 6.3 version.
Happy Snorting!
About the Multi-instance (at the moment) it”s ONLY available for Cisco Firepower 4100/9300 (too much $) will this feature be available on a Cisco ASA 5508 (as currently available a license with up 5 multiple context) ?
As the ASA5505 will be EoS cause will not support FTD version 6.3 do you know if a new “soho” Cisco Firepower applicance will be available or the Cisco ASA 5508 will be the new entry level device for FTD ?
Hardware: ASA5508, 8192 MB RAM, CPU Atom C2000 series 2000 MHz, 1 CPU (8 cores)
It will be eventually available on the 5500x but remember it’s not contexts but instances. That means each image gets its own CPU and memory. They can have a shared port, for example a port going to the internet, but that’s it, they are all very much separated services!
“It will be eventually available on the 5500x” – Are you really sure about that Todd? IMO multi-instance will be something that will be limited to FPR2100/4100/9300 since it is built on top of FX-OS hence there is a certain dependency. Combine that with the amount of resources required for multi-instance I think it will not be available on the old 5500-X series, which will probably be EOLED in a few years.
I had heard that the 5500x will get it, but we’ll see
they are all already EOL…but they can run a pure FTD image for a long time, so they might get it, we’ll see…but I don’t recommend running it on those for the reason you state
We have 16 firewalls split among two FMC appliances. Even with administering only 8 firewalls (FTDs and ASAs with Firepower), the system is extremely slow and sluggish. I can’t imagine trying to add multi-instance to this, or user-based access control.
The problem is largely with Java and the underlying system. Will Cisco ever go to HTML 5 in order to clean up FMC?
Yes, make sure you have 16 Gig or RAM if you’re vFMC….if you have 2500 or 4500 that’s the best you get
6.2.3 code should be your minimum code, if you’re not running that, it will be slower than need be.
Thanks for your attention.
Thanks for the write up… obviously the upgrade path is FMC then FTD… I’m curious if some of these features are available if you are only running 6.3 FMC and say 6.2.3.x for the FTD. Also, since the 5506s are no longer support beyond 6.2, will some of the administrative features available in 6.3 work for those devices? Namely the syslog configuration and the Contextual Cross-launch?
I run 6.3 FMC and 6.2.3 for FTD devices at a lot of customers right now – works great. Yes, you lose some features like FQDN but not Syslog…no problems so far and I have done about 40 devices in the last 1.5 weeks…thanks for writing!
Hi Todd, thanks for the write up.
Regarding the multi-instance.. do you know how many instances are supported on the 4100 appliances ?
-can’t seem to find the info in release notes.
Kind regards
Hi Stefan. Yes, you can get as many instances as you have cores. Each instance takes 8 cores, so depending on the model and cores you have will determine that amount
however, they are coming out with 4115, 4125, 4145 and 4155 soon and it will double the amount of cores!
Thanks Tod for new updates. Is it safe to upgrade FTD to 6.3? Is it stable ?
Very! I highly recommend it!
I’ve had 5 classes with it and most of my customers use it now too.
Hi Lammle, IAB and AAB are not enabled by default in FTD 6.3. no mention for that in release notes, also in the configuration guides its clearly stated that AAB is not enabled by default however your screen shots shows enabled:
“By default the AAB is disabled; to enable AAB follow the steps described.
Caution
AAB activates when an excessive amount of time is spent processing a single packet. AAB activation partially restarts the Snort process, which temporarily interrupts the inspection of a few packets. Whether traffic drops during this interruption or passes without further inspection depends on how the target device handles traffic. See Snort® Restart Traffic Behavior for more information”
https://www.cisco.com/c/en/us/td/docs/security/firepower/630/configuration/guide/fpmc-config-guide-v63/device_management_basics.html
“Not all deployments require IAB, and those that do might use it in a limited fashion. Do not enable IAB unless you have expert knowledge of your network traffic, especially application traffic, and system performance, including the causes of predictable performance issues. Before you run IABin bypass mode, make sure that trusting the specified traffic does not expose you to risk.”
https://www.cisco.com/c/en/us/td/docs/security/firepower/630/configuration/guide/fpmc-config-guide-v63/intelligent_application_bypass.html?bookSearch=true#task_7847C33C38C94298A8F13A0112EE5B6D
is it possible to adjust this information so its not misleading anyone. Thank you for your valuable contributions
yes, this is correct, Shahad, I believe I wrote that because it was supposed to be on by default, but they are not. AAB should be on by default, but even with 6.4 it still is not!
Thanks for posting!
Quick question regarding throughput and multi instance.
I’m currently setting up a 2 instance deployment on the 4110’s, on 10 Core and one 12 Core instance, and as far as I can tell I’m getting half the announced IPS throughput with empty ACP and default balanced security and connectivity.
To verify I assigned one instance with all 22 cores and I get maximum 5Gb/s throughput with all snort processes @ 100%
Does enabling multi instance actually cut my IPS througput in half? or am I doing it wrong?
Cant find anything about this particular aspect of multi instance anywhere.
it can cut it in half, what is your IPS policy set at, meaning do you have a default set?
that’s a little high on the latency though regardless
I just went with default balanced security and connectivity, and followed the best practices in the “Essential Firepower” book by Alex Tatistcheff. Enabled inline normalization in the NAP.
I actually found some info about this in BRKSEC-3035 by cisco live on-demand by Andrew Ossipov.
https://www.ciscolive.com/global/on-demand-library.html?search=firepower&search.event=ciscoliveus2019#/session/1542224329879001ro1t
In the slide deck he tells us how to calculate the throughput based on number of cores and whitepaper numbers.
• Maximum container instance throughput is proportional to CPU core count
• Step 1: Obtain maximum native instance (full cores) throughput from data sheet
• Step 2: Divide figure from Step 1 by native Snort cores
• Step 3: Multiply figure in Step 2 by Snort cores for instance size
Which, in my case for a 12 core instance results in (11Gbs / 12 Native Snort Cores) * 6 Instance Snort Cores = 5,5 Gb/s
I get only half that with 2,3 Gb/s. Which I find weird as single flow tests gives me 700+ Mb/s.
TAC justs says its not viable to test throughput and links to a Cisco live session describing how hard it is to do real tests on these types of devices, and I agree its not trivial if you want to really test.
But I would think that filling up the Snort Cores with X amount of large flows like iPerf sessions/SMB filetransfers/web downloads/etc from multiple machines would get us close to the maximum.
There are a number of factors that enter in when determining FTD performance. On the Snort side the major features are:
Base (AVC)
Threat (IPS)
Malware (AMP)
SSL Decryption
Enabling some or all of these will have a massive impact on the throughput of the device. According to Cisco, your FP4110 will be hammered at 100% if you push 4Gbps with Base, Threat and Malware enabled. Remove Malware and you’re still at 60% so you can see this one feature has a pretty major impact. We won’t even talk about SSL Decryption…
That being said, Docker architecture of multi-instance doesn’t introduce any more overhead on the Snort processes. However, you will lose 2 CPU cores to management with each instance. Because of this, your total multi-instance throughput won’t add up to the native device throughput. This gets worse with smaller devices (the 4110 being the smallest device that supports multi-instance) and with more instances.
To compare apples to apples you should use the “show snort instance” command from the CLI of your device. This is the FTD management IP assigned to the logical device/instance. This will tell you how many CPU cores are dedicated to Snort in that device. You can then have some idea what throughput you can expect.
Note that the CPU cores in an instance should be:
2 cores for overhead
50% cores for ASA
50% cores for Snort
Check this out and verify it’s what you’re seeing.
Great answer, Alex! thank you!
Thank you!
Good and detailed answer that confirms most of what I have learned so far. Really appreciate you guys taking the time to answer some random guys questions on the Internet.
I did not intend to create a “support thread” here, just to ask a quick question that I could not find good answers to anywhere else… I just hate not knowing exactly how things work and can’t let stuff go until I do…
I am curious though where do you get the 4 Gbps limit with Base, Threat and Malware?
From the official data sheet I get Throughput: FW + AVC + IPS (1024B) 11 Gbps, Can’t find the numbers for FW + AVC + IPS + AMP
I can confirm that with 12 cores assigned to the instance I have 6 Snort instances available
which does not add up to 2 cores management, 5 cores LINA, 5 cores Snort?
> show snort instances
Total number of instances available – 6
+———-+———+
| INSTANCE | PID |
+———-+———+
| 1 | 25233 |
| 2 | 25234 |
| 3 | 25235 |
| 4 | 25236 |
| 5 | 25237 |
| 6 | 25238 |
+———-+———+
The 4Gbps number comes from the Performance Estimator. It’s a tool that’s internally available at Cisco and designed primarily for pre-sales to get a good idea of what devices to scope for a particular use-case.
That’s interesting that you got 6 Snort cores out of a 12 core instance. That’s a good thing! I haven’t tested this myself but was going from some early training on multi-instance. It’s possible this was changed or my info wasn’t 100% accurate. I have noticed the two management cores when setting up logical devices on a FP9300. For example a SM-44 which has 88 cores shows 86 usable when you start assigning cores to instances.
Happy Snorting!