The Problem: 3CX Firewall Checker Failures

When deploying a 3CX PhoneSystem behind a FortiGate firewall, you might encounter issues with the firewall checker failing tests like "testing 3CX PhoneSystem 01 SIP Server... failed" or "detecting SIP ALG... failed." These failures can prevent proper SIP communication, leading to call drops or registration issues.

In this guide, I’ll walk you through how we resolved these issues on a FortiGate-VM64-KVM running FortiOS 7.4.7, ensuring all firewall checker tests passed and the 3CX system worked flawlessly.

Initial Firewall Checker Output

We started with this output from the 3CX firewall checker:

resolving 'stun-eu.3cx.com'... done
resolving 'stun2.3cx.com'... done
resolving 'stun3.3cx.com'... done
resolving 'sip-alg-detector.3cx.com'... done
testing 3CX PhoneSystem 01 SIP Server... failed
stopping service... done
detecting SIP ALG... failed
testing port 5060... done
starting service... done
testing 3CX PhoneSystem Media Server... done
stopping service... done
testing port 5090... done
testing ports [9000..9398]... done
        

The key failures were:

• SIP Server Test Failure: Indicating potential issues with SIP traffic handling on port 5060.
• SIP ALG Detection Failure: Suggesting either SIP ALG interference or connectivity issues with sip-alg-detector.3cx.com.

Step 1: Verify FortiOS Version

First, we confirmed the FortiOS version to tailor our commands correctly:

FGVMSLTM24005279 # get system status
Version: FortiGate-VM64-KVM v7.4.7,build2731,250120 (GA.M)
Operation Mode: NAT
Current HA mode: standalone
System time: Wed Mar 12 03:31:03 2025
        

FortiOS 7.4.7 meant we needed to use modern commands for disabling SIP ALG and session helpers.

Step 2: Remove the SIP Session Helper

FortiGate firewalls often have session helpers enabled by default, including one for SIP, which can modify SIP packets. We checked for active session helpers:

FGVMSLTM24005279 # config system session-helper
show
edit 13
    set name sip
    set protocol 17
    set port 5060
next
        

We removed this SIP session helper (ID 13):

config system session-helper
delete 13
end
        

Step 3: Disable SIP ALG via VoIP Profile

In FortiOS 7.4, SIP ALG is managed through VoIP profiles. We disabled it:

config voip profile
edit default
config sip
set status disable
set rtp disable
end
end
        

Some commands like set sip-nat-trace disable failed in this context due to version-specific changes in FortiOS 7.4.7.

Step 4: Disable SIP NAT Trace Globally

Since sip-nat-trace wasn’t available in the VoIP profile, we disabled it globally:

config system settings
set sip-nat-trace disable
end
        

Step 5: Clear Sessions and Reboot

We cleared sessions and rebooted the FortiGate:

diagnose sys session clear
execute reboot
        

Step 6: Verify Settings and Rerun the Firewall Checker

After rebooting, we confirmed the settings:

config voip profile
edit default
show
    set status disable
    set rtp disable
        

Then reran the 3CX firewall checker:

resolving 'stun-eu.3cx.com'... done
resolving 'stun2.3cx.com'... done
resolving 'stun3.3cx.com'... done
resolving 'sip-alg-detector.3cx.com'... done
testing 3CX PhoneSystem 01 SIP Server... done
stopping service... done
detecting SIP ALG... not detected
testing port 5060... done
starting service... done
testing 3CX PhoneSystem Media Server... done
stopping service... done
testing port 5090... done
testing ports [9000..9398]... done
        

All tests passed!

Why It Worked

The root cause was likely the SIP session helper and residual SIP ALG settings. Disabling them systematically resolved the issue.

Frequently Asked Questions

What is SIP ALG, and why disable it for 3CX?

SIP ALG (Application Layer Gateway) modifies SIP packets to help with NAT traversal, but it often causes issues with 3CX by altering packets in unexpected ways. Disabling it ensures 3CX handles SIP traffic directly.

Why did the SIP Server test fail initially?

The SIP Server test likely failed due to FortiGate's session helper or SIP ALG modifying packets, preventing proper SIP communication on port 5060.

What if my firewall checker still fails after these steps?

Double-check NAT and firewall policies, ensure connectivity to 3CX servers (e.g., sip-alg-detector.3cx.com), and capture packets using diagnose sniffer packet any "port 5060" 4 to identify issues.

Do I need to update FortiOS to resolve 3CX issues?

Not always. This guide worked with FortiOS 7.4.7, but if issues persist, check Fortinet’s release notes for known SIP bugs in your version and consider updating to a stable release.