Fortigate Firewall Traffic Processing Troubleshooting

Traffic Processing

Application Debugging

diag debug application 

shows what happens during the execution of a process. Detailed information is shown. Debug level is a bit mask.

diag debug application [application name] [debug level]

Debug level: -1 or 255 displays everything (normally). 

It is possible to show date and time:

diag debug console timestamp enable

Start or stop to display the output of the information.

diag debug enable
diag debug disable

Reset all debugging to 0

diag debug reset

 

diag test application 

this command prints settings and status of processes. Additionally it is possible to modify these settings or to change the behaviour.

diag test application [application name] [test]

Using Test Level “0” prints usually a help page.

Example with ipsmonitor:

diag test application ipsmonitor 0

will show

2018-03-10 08:09:00 
IPS Engine Test Usage:

2018-03-10 08:09:00     1: Display IPS engine information
2018-03-10 08:09:00     2: Toggle IPS engine enable/disable status
2018-03-10 08:09:00     3: Display restart log
2018-03-10 08:09:00     4: Clear restart log
2018-03-10 08:09:00     5: Toggle bypass status
2018-03-10 08:09:00     6: Submit attack characteristics now
2018-03-10 08:09:00    10: IPS queue length
2018-03-10 08:09:00    11: Clear IPS queue length
2018-03-10 08:09:00    12: IPS L7 socket statistics
2018-03-10 08:09:00    13: IPS session list
2018-03-10 08:09:00    14: IPS NTurbo statistics
2018-03-10 08:09:00    15: IPSA statistics
2018-03-10 08:09:00    16: Display device identification cache
2018-03-10 08:09:00    17: Clear device identification cache
2018-03-10 08:09:00    18: Display session info cache
2018-03-10 08:09:00    19: Clear session info cache
2018-03-10 08:09:00    21: Reload FSA malicious URL database
2018-03-10 08:09:00    22: Reload whitelist URL database
2018-03-10 08:09:00    24: Display Flow AV statistics
2018-03-10 08:09:00    25: Reset Flow AV statistics
2018-03-10 08:09:00    96: Toggle IPS engines watchdog timer
2018-03-10 08:09:00    97: Start all IPS engines
2018-03-10 08:09:00    98: Stop all IPS engines
2018-03-10 08:09:00    99: Restart all IPS engines and monitor

Careful: Using diag test application can have severe impacts. Use it first on a test system, if you do not know what you do.

Packet Sniffer

Every FortiGate got a built in sniffer

diag sniffer packet [interface] '[filter]' [verbose level] [count] [tsformat]

Interface: ‘any’ or the name of an interface

Filter: corresponds with the ⇒libcap format like tcpdump.

Verbose level:

1packet header
2packet header, IP data
3packet header, IP data, ethernet header
4interface, direction, packet header
5interface, direction, packet header, IP data
6interface, direction, packet header, IP data, ethernet header

 

Count: 0 – unlimited packets or a number of packets

Tsformat: a or l, GMT or localtime. If tsformat is omitted, number of seconds since program start will be shown.

Careful: If a policy is accelerated, only the first and the last packet is shown. It is possible to disable hardware acceleration in a firewall policy.

config firewall policy
  edit <id>
    set auto-asic-offload disable
  next
end

Please reactivate acceleration again after your troubleshooting session!

Flow Trace

The sniffer shows you the packets at the interface. diag debug flow shows you, what happens in between. With the flow trace you can find out what exactly blocks the traffic.

Clear possible filters from a previous session

diag debug flow filter clear

Limit the traffic to specific filters

diag debug flow filter [filter]

Show the function name

diag debug flow show function-name enable

Start the output on the terminal

diag debug enable

Start the trace

daig debug flow trace start [packet count]

Packet count: 0 – unlimited number of packets, or a specific number of packets.

Stop the trace

diag debug flow trace stop

 

Session Table Troubleshooting

Clear possible filters

diag sys session filter clear

Limit the output to specific sessions

diag sys session filter [filter]

The filter entries are cumulated. You can set multiple filters like address, port, interface.

Show the filters set

diag sys session filter

Show the actual sessions limited by the filter

diag sys session list [expect]

expect will show you temporary sessions created by a session helper.

Clear the sessions limited by the filter

diag sys session clear

Only the sessions that match the filter will be cleared. If no filter is specified, all sessions will be cleared.

Get some session statistics

diag sys session stat

 

The command “diag sys session list” uses the interface indices instead of the names. Using

diagnose netlink interface list

the name and the index of the interface is shown.

 

You can reset the counters of all or of specific firewall polices on the  CLI

diag firewall iprope clear 00100004 [<id>]