docsis_server How To

Setting up Cisco CMTS with Virtual Routing and Forwarding (VRF)

Cisco Virtual Routing and Forward is a type of VLAN except its done with routing tables. The CMTS has a regular routing table, but it can also have seperate Virtual routing tables that allow you to segment traffic. VRF can be tricky to set up. I recommend consultation with the Cisco TAC. Try to get past the front line cable guys though. The one problem that crops up is that it is very difficult to leak routes between the global routing table and the virtual tables. If most of your modems are going to be in one routing table, you probably want it to be the global table. For that reason (and with Cisco's recomendation) I do all my route leaks on a separate Cisco 3620 router.

version 12.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
!
hostname docsis-example1
!
boot system flash disk2:ubr7200-ik8s-mz.122-15.BC2e.bin
boot bootldr disk2:ubr7200-boot-mz.122-15.BC2e.bin
logging queue-limit 100
logging buffered 16000 debugging
no logging console
enable secret secret-enable
!
username letmein password letmein
cable modem max-cpe 4
cable spectrum-group 1 shared
cable spectrum-group 1 frequency 25000000
cable spectrum-group 1 frequency 32000000
cable spectrum-group 2 shared
cable spectrum-group 2 frequency 25000000
cable spectrum-group 2 frequency 32000000
cable spectrum-group 3 shared
cable spectrum-group 3 frequency 25000000
cable spectrum-group 3 frequency 32000000
cable modulation-profile 142 request 0 16 0 8 qpsk scrambler 152 no-diff 64 fixed
cable modulation-profile 142 initial 5 34 0 48 qpsk scrambler 152 no-diff 128 fixed
cable modulation-profile 142 station 5 34 0 48 qpsk scrambler 152 no-diff 128 fixed
cable modulation-profile 142 short 3 78 35 25 qpsk scrambler 152 no-diff 100 shortened
cable modulation-profile 142 long 9 232 0 137 qpsk scrambler 152 no-diff 80 shortened
cable modulation-profile 142 a-short 3 78 35 25 qpsk scrambler 152 no-diff 100 shortened qpsk0 1 2048
cable modulation-profile 142 a-long 9 232 0 137 qpsk scrambler 152 no-diff 80 shortened qpsk0 1 2048
cable modulation-profile 142 a-ugs 9 232 0 137 qpsk scrambler 152 no-diff 80 shortened qpsk0 1 2048
cable modulation-profile 143 request 0 16 0 8 qpsk scrambler 152 no-diff 64 fixed
cable modulation-profile 143 initial 5 34 0 48 qpsk scrambler 152 no-diff 128 fixed
cable modulation-profile 143 station 5 34 0 48 qpsk scrambler 152 no-diff 128 fixed
cable modulation-profile 143 short 3 78 35 25 qpsk scrambler 152 no-diff 100 shortened
cable modulation-profile 143 long 9 232 0 137 qpsk scrambler 152 no-diff 80 shortened
cable modulation-profile 143 a-short 5 78 19 17 16qam scrambler 152 no-diff 100 shortened qpsk1 1 2048
cable modulation-profile 143 a-long 9 232 139 77 16qam scrambler 152 no-diff 108 shortened qpsk1 1 2048
cable modulation-profile 143 a-ugs 9 232 139 77 16qam scrambler 152 no-diff 108 shortened qpsk1 1 2048
cable qos profile 5 max-burst 1600
cable qos profile 5 max-downstream 1024
cable qos profile 5 max-upstream 1024
cable qos profile 5 priority 1
cable qos profile 5 tos-overwrite 0xFF 0x0
no cable qos permission create
no cable qos permission update
cable qos permission modems
aaa new-model
!
!
aaa authentication login default local
aaa session-id common
ip subnet-zero
no ip source-route
no ip icmp rate-limit unreachable
ip wccp version 1
!
ip nbar pdlm disk2:kazaa2.pdlm
ip nbar pdlm disk2:gnutella.pdlm
!
ip cef
ip domain name example.com
ip name-server 24.27.0.6
ip name-server 24.27.0.7
ip dhcp relay information option
!
ip vrf a_staff
 rd 65010:13
 route-target export 65010:13
 route-target import 65010:13
 route-target import 65010:11
!
ip vrf b_customer
 rd 65010:27
 route-target export 65010:27
 route-target import 65010:27
 route-target import 65010:11
!
ip vrf c_customer
 rd 65010:12
 route-target export 65010:12
 route-target import 65010:12
 route-target import 65010:11
!
ip vrf mgmt
 rd 65010:11
 route-target export 65010:11
 route-target import 65010:11
!
mpls ldp logging neighbor-changes
!
 class-map match-any p2p-out
  match access-group 112
  match protocol napster
  match protocol kazaa2 file-transfer "*"
  match protocol gnutella file-transfer "*"
!
 policy-map p2p-out
  class p2p-out
   police cir 3000000 bc 8000 be 8000
     conform-action transmit
     exceed-action drop
!
interface Loopback0
 ip address 10.0.0.3 255.255.255.255
!
interface Port-channel3
 no ip address
 load-interval 30
 duplex full
 no cdp enable
 hold-queue 150 in
!
interface Port-channel3.10
 description global internet
 encapsulation dot1Q 10
 ip address 24.27.0.10 255.255.255.0
 service-policy output p2p-out
 no cdp enable
!
interface Port-channel3.11
 description mgmt vrf
 encapsulation dot1Q 11
 ip vrf forwarding mgmt
 ip address 10.1.1.5 255.255.255.0
 no cdp enable
!
interface Port-channel3.12
 description c_customer vrf
 encapsulation dot1Q 12
 ip vrf forwarding c_customer
 ip address 10.100.1.253 255.255.255.0
 no cdp enable
!
interface Port-channel3.13
 description a_staff vrf
 encapsulation dot1Q 13
 ip vrf forwarding a_staff
 ip address 192.168.4.253 255.255.255.0
 no cdp enable
!
interface Port-channel3.27
 description b_customer VRF
 encapsulation dot1Q 27
 ip vrf forwarding b_customer
 ip address 192.168.8.9 255.255.255.248
 no cdp enable
!
interface GigabitEthernet0/1
 description EtherChannel interface #1 to 7606
 no ip address
 load-interval 30
 duplex full
 speed 100
 media-type rj45
 no negotiation auto
 channel-group 3
 no cdp enable
!
interface GigabitEthernet0/2
 description EtherChannel interface #2 to 7606
 no ip address
 load-interval 30
 duplex full
 speed 100
 media-type rj45
 no negotiation auto
 channel-group 3
 no cdp enable
!
!
interface Cable3/0
 description Fibre Hubs 55, 23, 19
 no ip address
 no ip unreachables
 cable shared-secret secret-shared
 cable bundle 1 master
 cable downstream annex B
 cable downstream modulation 64qam
 cable downstream interleave-depth 8
 cable downstream frequency 567000000
 cable downstream channel-id 0
 cable upstream 0 description Hub 55 - p1
 cable upstream 0 spectrum-group 1
 cable upstream 0 ingress-noise-cancellation 200
 cable upstream 0 power-level 14
 cable upstream 0 channel-width 3200000 3200000
 cable upstream 0 minislot-size 2
 cable upstream 0 range-backoff 3 6
 cable upstream 0 modulation-profile 41
 no cable upstream 0 shutdown
 cable upstream 1 description Hub 55 - p2
 cable upstream 1 spectrum-group 1
 cable upstream 1 ingress-noise-cancellation 200
 cable upstream 1 power-level 14
 cable upstream 1 channel-width 3200000 3200000
 cable upstream 1 minislot-size 2
 cable upstream 1 range-backoff 3 6
 cable upstream 1 modulation-profile 41
 no cable upstream 1 shutdown
 cable upstream 2 description Hub 23
 cable upstream 2 frequency 32000000
 cable upstream 2 ingress-noise-cancellation 200
 cable upstream 2 power-level 14
 cable upstream 2 channel-width 3200000 3200000
 cable upstream 2 minislot-size 2
 cable upstream 2 range-backoff 3 6
 cable upstream 2 modulation-profile 41
 cable upstream 2 s160-atp-workaround
 no cable upstream 2 shutdown
 cable upstream 3 description Hub 19
 cable upstream 3 frequency 32000000
 cable upstream 3 ingress-noise-cancellation 200
 cable upstream 3 power-level 14
 cable upstream 3 channel-width 3200000 3200000
 cable upstream 3 minislot-size 2
 cable upstream 3 range-backoff 3 6
 cable upstream 3 modulation-profile 41
 cable upstream 3 s160-atp-workaround
 no cable upstream 3 shutdown
 no cable ip-multicast-echo
 cable source-verify dhcp
 cable source-verify leasetimer 240
 cable dhcp-giaddr primary
 no keepalive
!
interface Cable3/0.1
 description ****** Provisioning and Management *****
 ip vrf forwarding mgmt
 ip address 10.34.1.1 255.255.0.0
 cable dhcp-giaddr primary
 cable helper-address 10.1.1.50
!
interface Cable3/0.3
 description ***** Normal Customers VLAN ********
 ip address 24.27.1.0 255.255.255.0 secondary
 ip address 24.27.2.0.1 255.255.255.0 secondary
 ip address 10.35.1.1 255.255.0.0
 no ip unreachables
 cable source-verify dhcp
 cable dhcp-giaddr primary
 cable helper-address 10.1.1.50
!
interface Cable3/0.5
 description ***** Customer C VLAN ********
 ip vrf forwarding c_customer
 ip address 10.99.6.1 255.255.255.0 secondary
 ip address 10.99.7.1 255.255.255.0 secondary
 ip address 10.36.1.1 255.255.0.0
 cable ip-broadcast-echo
 cable dhcp-giaddr primary
 cable helper-address 10.1.1.50
!
interface Cable3/0.7
 description ***** Staff VLAN *******
 ip vrf forwarding a_staff
 ip address 24.27.65.65 255.255.255.0 secondary
 ip address 10.39.1.1 255.255.0.0
 no ip unreachables
 cable source-verify dhcp
 cable dhcp-giaddr primary
 cable helper-address 10.1.1.50
!
interface Cable3/0.9
 description ***** customer B VLAN *********
 ip vrf forwarding b_customer
 ip address 192.168.8.1 255.255.255.248 secondary
 ip address 10.40.1.1 255.255.0.0
 no ip unreachables
 cable dhcp-giaddr primary
 cable helper-address 10.1.1.50
!
interface Cable3/1
 description Fibre Hubs 17, 18
 no ip address
 no ip unreachables
 ip route-cache flow
 cable shared-secret secret-shared
 cable bundle 1
 cable downstream annex B
 cable downstream modulation 64qam
 cable downstream interleave-depth 8
 cable downstream frequency 567000000
 cable downstream channel-id 1
 cable upstream 0 description Hub 17 - p1
 cable upstream 0 spectrum-group 2
 cable upstream 0 ingress-noise-cancellation 200
 cable upstream 0 power-level 14
 cable upstream 0 channel-width 3200000 3200000
 cable upstream 0 minislot-size 2
 cable upstream 0 range-backoff 3 6
 cable upstream 0 modulation-profile 41
 no cable upstream 0 shutdown
 cable upstream 1 description Hub 17 - p2
 cable upstream 1 spectrum-group 2
 cable upstream 1 ingress-noise-cancellation 200
 cable upstream 1 power-level 14
 cable upstream 1 channel-width 3200000 3200000
 cable upstream 1 minislot-size 2
 cable upstream 1 range-backoff 0 6
 cable upstream 1 modulation-profile 41
 no cable upstream 1 shutdown
 cable upstream 2 description Hub 18 - p1
 cable upstream 2 spectrum-group 3
 cable upstream 2 ingress-noise-cancellation 200
 cable upstream 2 power-level 14
 cable upstream 2 channel-width 3200000 3200000
 cable upstream 2 minislot-size 2
 cable upstream 2 range-backoff 3 6
 cable upstream 2 modulation-profile 41
 no cable upstream 2 shutdown
 cable upstream 3 description Hub 18 - p2
 cable upstream 3 spectrum-group 3
 cable upstream 3 ingress-noise-cancellation 200
 cable upstream 3 power-level 14
 cable upstream 3 channel-width 3200000 3200000
 cable upstream 3 minislot-size 2
 cable upstream 3 range-backoff 3 6
 cable upstream 3 modulation-profile 41
 no cable upstream 3 shutdown
 no cable ip-multicast-echo
 cable source-verify dhcp
 cable dhcp-giaddr primary
 no keepalive
!
router bgp 65010
 no synchronization
 bgp log-neighbor-changes
 redistribute connected
 redistribute static
 no auto-summary
 !
 address-family ipv4 vrf c_customer
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf mgmt
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf b_customer
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf a_staff
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
!
ip classless
ip route 0.0.0.0 0.0.0.0 24.27.0.1
ip route 10.1.1.0 255.255.255.0 24.27.99.250
ip route 24.27.3.16 255.255.255.255 24.27.1.27
ip route vrf a_staff 0.0.0.0 0.0.0.0 192.168.4.1
ip route vrf b_customer 0.0.0.0 0.0.0.0 192.168.8.11
ip route vrf b_customer 10.53.16.128 255.255.255.128 192.168.8.2
ip route vrf c_customer 0.0.0.0 0.0.0.0 10.100.1.1
ip route vrf c_customer 10.1.1.0 255.255.255.0 10.100.1.250
no ip http server
ip http authentication local
no ip http secure-server
!
!
!
logging trap debugging
logging facility local4
logging 10.1.1.50
access-list compiled
access-list 1 permit 10.1.1.50
access-list 99 deny   any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any range 411 412
access-list 112 permit tcp 24.27.0.0 0.0.255.255 range 411 412 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 1214
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 1214 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 4662
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 4662 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 6347
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 6347 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 6688
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 6688 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 6699
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 6699 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 2234
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 2234 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any eq 5534
access-list 112 permit tcp 24.27.0.0 0.0.255.255 eq 5534 any
access-list 112 permit tcp 24.27.0.0 0.0.255.255 any range 6880 6890
access-list 112 permit tcp 24.27.0.0 0.0.255.255 range 6880 6890 any
access-list 199 permit icmp any any echo
access-list 199 permit icmp any any echo-reply
no cdp run
!
snmp-server community readonly-name RO 1
snmp-server enable traps tty
!
radius-server authorization permit missing Service-Type
banner motd ^C
Welcome to docsis-example1^C
alias exec spc show proc cpu | exclude 0.00
alias exec bping ping vrf b_customer
alias exec cping ping vrf c_customer
!
line con 0
 stopbits 1
line aux 0
 stopbits 1
line vty 0 4
 session-timeout 360
 access-class 1 in
 exec-timeout 360 0
line vty 5 15
 session-timeout 360
 access-class 1 in
 exec-timeout 360 0
!
exception core-file 7246-file
exception protocol ftp
exception dump 24.27.0.50
scheduler allocate 4000 400
ntp clock-period 17180048
ntp server 24.27.0.50
end

This config also uses ether-channel to combine 2 100 mbps ports. You need to connect the CMTS to a switch capable of 802.1q ethernet vlans. You also need to connect the 3620 cisco router to the switch to leak routes. Of course you also need another router to actually connect your CMTS to the Internet.
This is an example of a config with only 2 downstream with 8 upstreams. I have highlighted a few options that are important for the docsis_server.

ip dhcp relay information option - this tells the CMTS to tack on the MAC address of the CM to any DHCP packets that come through. This enables you to tell which CM a CPE dhcp request is coming from.
cable shared-secret shared-secret - don't forget to secure your config files with a secret shared key.
cable source-verify dhcp - If a CPE packet comes in and the CMTS doesn't have a record of the IP/MAC combination it will try to verify the IP with the docsis_server.
cable dhcp-giaddr primary - This option sets the GIADDR field of the DHCP packets that get relayed by the CMTS. If you use "primary" then the "cmts_ip" field of the "config_nets" table should be the primary IP address of the Cable interface. If you use "secondary" then the "cmts_ip" field of the "config_nets" table should be gateway IP address for that network.
cable helper-address 24.27.0.5 - This should be the IP address of your docsis_server.
logging 24.27.0.5 - Allow logging of errors to the docsis_server.

version 12.2
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname c3620-vrf
!
boot system flash slot0:c3620-is-mz.122-15.T12.bin
logging queue-limit 100
enable secret really secret
!
username letmein password letmein
aaa new-model
!
!
aaa authentication login default local
aaa session-id common
ip subnet-zero
!
!
ip cef
ip domain name example.com
ip name-server 24.27.0.6
!
ip vrf a_staff
 rd 65010:13
 route-target export 65010:13
 route-target import 65010:13
 route-target import 65010:11
!
ip vrf b_customer
 rd 65010:27
 route-target export 65010:27
 route-target import 65010:27
 route-target import 65010:11
!
ip vrf c_customer
 rd 65010:12
 route-target export 65010:12
 route-target import 65010:12
 route-target import 65010:11
!
ip vrf internet
 rd 65010:10
 route-target export 65010:10
 route-target import 65010:10
 route-target import 65010:11
!
ip vrf mgmt
 rd 65010:11
 route-target export 65010:11
 route-target import 65010:11
 route-target import 65010:12
 route-target import 65010:10
 route-target import 65010:13
 route-target import 65010:27
!
no voice hpi capture buffer
no voice hpi capture destination
mta receive maximum-recipients 0
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface FastEthernet0/0
 no ip address
 duplex auto
 speed auto
 no cdp enable
!
interface FastEthernet0/0.10
 description Internet
 encapsulation dot1Q 10
 ip vrf forwarding internet
 ip address 24.27.0.2 255.255.255.0
 no cdp enable
!
interface FastEthernet0/0.11
 description mgmt VRF
 encapsulation dot1Q 11
 ip vrf forwarding mgmt
 ip address 10.1.1.2 255.255.255.0
 no cdp enable
!
interface FastEthernet0/0.12
 description c_customer VRF
 encapsulation dot1Q 12
 ip vrf forwarding c_customer
 ip address 10.100.1.250 255.255.255.0
 no cdp enable
!
interface FastEthernet0/0.13
 description a_staff vrf
 encapsulation dot1Q 13
 ip vrf forwarding a_staff
 ip address 192.168.4.250 255.255.255.0
 no cdp enable
!
interface FastEthernet0/0.27
 description b_customer VRF
 encapsulation dot1Q 27
 ip vrf forwarding b_customer
 ip address 192.168.8.10 255.255.255.248
 no cdp enable
!
router bgp 65010
 no synchronization
 bgp log-neighbor-changes
 redistribute connected
 redistribute static
 no auto-summary
 !
 address-family ipv4 vrf c_customer
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf mgmt
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf internet
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf b_customer
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
 !
 address-family ipv4 vrf a_staff
 redistribute connected
 redistribute static
 no auto-summary
 no synchronization
 exit-address-family
!
no ip http server
ip classless
ip route vrf a_staff 0.0.0.0 0.0.0.0 192.168.4.1
ip route vrf a_staff 10.37.0.0 255.255.0.0 192.168.4.251
ip route vrf a_staff 10.38.0.0 255.255.0.0 192.168.4.252
ip route vrf a_staff 10.39.0.0 255.255.0.0 192.168.4.253
ip route vrf a_staff 24.27.65.0 255.255.255.224 192.168.4.251
ip route vrf a_staff 24.27.65.32 255.255.255.224 192.168.4.252
ip route vrf a_staff 24.27.65.64 255.255.255.224 192.168.4.253
ip route vrf b_customer 0.0.0.0 0.0.0.0 192.168.8.11
ip route vrf b_customer 10.40.0.0 255.255.0.0 192.168.8.9
ip route vrf c_customer 0.0.0.0 0.0.0.0 10.100.1.1
ip route vrf c_customer 10.13.0.0 255.255.0.0 10.100.1.254
ip route vrf c_customer 10.16.0.0 255.255.0.0 10.100.1.251
ip route vrf c_customer 10.33.0.0 255.255.0.0 10.100.1.252
ip route vrf c_customer 10.36.0.0 255.255.0.0 10.100.1.253
ip route vrf internet 0.0.0.0 0.0.0.0 24.27.99.249
ip route vrf internet 10.12.0.0 255.255.0.0 24.27.99.254
ip route vrf internet 10.15.0.0 255.255.0.0 24.27.99.251
ip route vrf internet 10.32.0.0 255.255.0.0 24.27.99.252
ip route vrf internet 10.35.0.0 255.255.0.0 24.27.99.253
ip route vrf internet 24.27.0.0 255.255.0.0 24.27.99.249
ip route vrf internet 24.27.50.0 255.255.255.0 24.27.99.251
ip route vrf internet 24.27.117.0 255.255.255.0 24.27.99.252
ip route vrf internet 24.27.103.0 255.255.255.0 24.27.99.253
ip route vrf mgmt 10.11.0.0 255.255.0.0 10.1.1.9
ip route vrf mgmt 10.14.0.0 255.255.0.0 10.1.1.3
ip route vrf mgmt 10.31.0.0 255.255.0.0 10.1.1.4
ip route vrf mgmt 10.34.0.0 255.255.0.0 10.1.1.5
!
!
!
access-list 1 permit 24.27.0.0 0.0.0.255
access-list 100 permit ip 192.168.3.0 0.0.0.255 any
access-list 101 permit ip any 192.168.3.0 0.0.0.255
no cdp run
!
radius-server authorization permit missing Service-Type
call rsvp-sync
!
!
mgcp profile default
!
dial-peer cor custom
!
!
!
alias exec spc show proc cpu | exclude 0.00
!
line con 0
line aux 0
line vty 0 4
 password 7 001016164E490A05
!
!
end

The idea behind this configuration is to leak routes from the other routing tables into the MGMT VRF. When a modem boots up the CMTS puts in into the MGMT VRF to begin with. The IP address the modem is assigned indicates which VRF the modem should be in. Your docsis_server should be in the MGMT VRF. The router leaks dhcp requests into the MGMT VRF.