Monday, April 24, 2017
 Now reading ...
Sep
23
Tue
Posted By Subodh on Tuesday, September 23, 2008
28859 Views 22 Comments


Mashup of ISA server 2006 and Microsoft Server 2008 Hyper VFor the uninitiated, Microsoft's Hyper V provides a compelling reason for consolidating multiple physical machines into fewer ones while still maintaining most if not all the features of a full fledged physical machine. Learn more about the technical details at the Technet Magazine page.

The reader may remember my obsession with staying green and trying to come up with solutions inside my home to reduce electricity consumption. As part of that plan, at least three machines, the domain controller, the web-server and the mail server were virtualized under Hyper-V.  It is a very straightforward process and is documented at various places. What is not documented however is, the final piece of the puzzle; Is it possible to virtualize ISA Server 2006 (edge firewall) inside the Hyper-V? If yes, does it still protect the host server?

 

The answer is "yes" to both the questions and the whole setup can be done in less than an hour. I'll keep the unimportant details like installing the Hyper V, basic installation of the ISA server etc out of this article. If the readers need more details, please let me know via the comments and I'd update the post.

 

Here's how the physical diagram of the setup looks like but before we go any further, its time for the usual "Disclaimer1"

The host Windows 2008 Server with Hyper V needs at least three network adapters. 

Assuming a simple home based network, the Router above represents the sole connection to the ISP's backbone (can be a DSL or a cable modem router) and is set as a transparent bridge aka - the ISA server will take the WAN IP addresses. Here's the run-down on the whole setup.

 

Step 1. Install the Windows Server 2008 on the host machine (with all the NICs onboard). For the installation part, connect any one network port to the internet for Windows Update. Once the installation is done, open up network adapters and rename them as follows.Figure 1: Hyper-V Virtual Network setup.

  • Physical Gateway - NIC0
  • Physical Public -NIC1
  • Physical Internal - NIC2

Step 2. Reboot the machine. Step 1 and 2 are required to make it easy to identify which network port is which. Trust me, you need the names to figure out and ensure you're connecting the right physical and virtual ports to the right places. A little mistake here and all your security is out of the way. If you do not reboot prior to step 3, you won't see the new names you assigned to each Local Area Connection.

 

Step 3.Install Hyper-V. During the installation, hyper-V will ask for the network adapter that you want to configure for virtual machine networking. Choose the "Physical Internal" network adapter you renamed in step 1. Installation of Hyper-V will require a reboot again.

 

Step 4. Windows Update. Requires Reboot. Before you reboot, remove all network connections and set your router to transparent bridge mode.

 

Step 5. Open up the Network Adapters again. Here you would find a new network adapter (virtual) that Hyper-V created for you. Rename this to "Virtual Internal". Open up Hyper-V management console. Fire up the Virtual Network Manager. Rename the in-use adapter to "Virtual Internal" as well.

 

Step 6. Using the Virtual Network Manager, create virtual adapters of type External for the remaining two adapters. See Figure 1 for illustration. Make sure you rename the Network Adapters as and when you create a new Virtual Adapter with the same name. Avoid confusion here and you'd save countless hours figuring out what went wrong.

 

Figure 2: Network adapters with both Virtual and Physical Adapters

Step 7. Virtual Network Manager view should look like Figure 1. Network Adapters should look something like Figure 2.

 

 

Step 8. Open up properties for all the three Virtual Adapters from the Network Connections control panel and change them as shown below.

 

Physical Gateway Network Adapter properties Virtual Gateway Network Adapter properties Virtual Internal Network Adapter properties Virtual Public Network Adapter properties

Step 9. Disconnect the ISP side of the router. Connect the ethernet port of router to NIC1 (Physical Gateway) via  a straight cable (assuming auto-sensing ports). You can check if you have connected to the right NIC by looking at the disconnected/connected icon in Control Panel-> Network Connections. Hopefully, if you named everything properly earlier, this should be a breeze.

Step 10. Connect the remaining two NICs on the Hyper-V to your network switch via standard CAT5/6 cables. Connections to all internal network begin and terminate from this switch and it forms a part of the secured network once ISA is installed.

Step 11. Create a new virtual machine with the following settings.

image

Step 12. Install Windows Server 2003 x86 on this machine and rename the two virtual adapters inside the ISA VM accordingly (you can test which one is which by disconnecting the router to NIC1 network cable and checking which network port shows up as disconnected)

image

Step 13. Assign the external WAN IP to the public interface on ISA server. Assign the internal gateway (for example 192.168.1.1) to the internal interface on ISA server. Configure the firewall as you normally would.

image

 

That's it! You're done. You can connect the router back to the ISP backbone and surf away while keeping everything inside your network secure.

Update 12/5/2008

To explain more as Anthony asked in comments below, here is an updated figure

 

Logical Setup Diagram

The Red Ethernet Cable (in the figure) from the Internet router goes to  Physical Public, the Host machine will not have access to it, the Hyper-V changes that (with our settings above) to Virtual Public and only the ISA uses this (thus firewalls). Example IP addresses: whatever your ISP has allocated for you for eg: 76.210.162.111/255.255.254.0 (The ISA Server uses these IPs on the WAN side)

The Green and the Blue Ethernet cables both go to the same network switch (which belongs to the internal network).

The Green CAT cable goes to port marked Physical gateway to which the Hyper-V host machine does not have access, is turned into Virtual Gateway (by our settings above)  and the ISA is the only machine that can use this.  (The ISA server acts as gateway and needs/uses two networks) Example IP address that one would normally assign: The gateway IP address for your internal chosen Private IP eg. 192.168.1.1/255.255.255.0 With this setting, the route to gateway/ISA is available at the internal network switch and any number of networks connected to this switch should (if set properly) be able to access the Internet via the ISA server.

The Blue Ethernet Cable  connects the internal network switch  to  Hyper-V machine (via Physical Internal) as well as to all the Virtual Machines (via Virtual Internal) running inside of the Hyper-V on this box. Example of VMs running on my box: The Domain controller, DHCP Server, DNS, etc… Note that the ISA is NOT connected to this network. Typically I would not assign an IP address unless its a server and let DHCP lease take-over from here but servers such as (VMs) Exchange Server, DNS, DHCP etc have their own static IPs such as 192.168.1.5/255.255.255.0 etc … in the same private IP range as our internal network.

Note that in the entire setup above, we did not use Hyper-V partitioning.

 

 

 

 

 

 

1

DISCLAIMER This posting is provided "AS IS" with no warranties, and confers no rights. You assume all risk for your use. Microsoft® is a registered trademark of Microsoft Corporation.

Categories

  
 You may also be interested in
  
 Comments & Discussions

  • Gravatar
    guest Thursday, September 25, 2008 at 5:36 PM
    Re: Running ISA Server 2006 under Hyper-V
    wow man this is awesome. Now why did you create the virtual internal? Its also connected to the switch so I'm assuming there is not need for either the switch or the virtual internal.

  • Gravatar
    Subodh Thursday, September 25, 2008 at 5:54 PM
    Re: Running ISA Server 2006 under Hyper-V
    The Virtual Internal Network is used for other virtual machines on the same host. The Virual gateway is left untouched and is meant only for the ISA virtual machine.

    • Gravatar
      Mansi Tuesday, July 07, 2009 at 12:42 PM
      Re: Running ISA Server 2006 under Hyper-V
      Hi Subodh,

      Thanks for the useful information.
      In my environment, ISA server 2006 is installed on Hyper-V machine. ISA server is having 2 NICS one connected to internal network i.e. AD and the other is external network [which is configured as Internal-Only virtual nic]. The issue i am facing is my client machine which is connected to external network is not able to ping ISA server. But from ISA server i am able to ping client machine (which is on external network).

      Can you please tell me whether pinging ISA server from client machine is really required? I basically want to check if there is some issue with my network configuration/Network adapters.

      -Thanks

      • Gravatar
        Subodh Tuesday, July 07, 2009 at 5:14 PM
        Re: Running ISA Server 2006 under Hyper-V
        It is considered a security risk to allow ping (ICMP traffic) to the ISA Server from the external network. I would also strongly advise against such a practice. You can however enable select machines to be part of Remote Management Computers (from System Policy) and enable ICMP/Ping from these machines alone.

        Internal machines are okay (and should be able) to ping the ISA irrespective.

  • Gravatar
    camboy Friday, September 26, 2008 at 1:01 AM
    How sure are you host is protected????
    Dude I gotta be 100% sure this will work. I do not believe you unless I find it on MS site. You seem to be saying windows WILL NOT route to the Host machine even if it is physically connected! What about IP fragments or other kind of crap that comes through?

  • Gravatar
    Subodh Friday, September 26, 2008 at 1:30 AM
    Re: How sure are you host is protected????
    I have just described how ISA can be installed and made to work under Hyper-V. For the absolute source of truth, please ask yourself here or post at the newsgroups. I did request for a confirmation of the facts before this post went live; unfortunately I haven't heard from Ben yet.

  • Gravatar
    Mark Saturday, September 27, 2008 at 1:34 AM
    Re: Running ISA Server 2006 under Hyper-V
    Did you need to make any changes to the registry? What are the DNS entries for ISA server like ? Where is your internal DNS server running? Is ISA running it? sorry, too many questions but it would help me a lot.

  • Gravatar
    Subodh Monday, September 29, 2008 at 5:41 PM
    Re: Running ISA Server 2006 under Hyper-V
    No, the sole reason for using three NICs is so that I do not have to hack around the registry and the whole setup remains portable.
    The DNS entries for the ISA server are : External (public) interface points to ISP DNS server; Internal(private) interface points to internal DNS server.
    The DNS server runs on another VM inside the network.

  • Gravatar
    ThomasL Wednesday, October 15, 2008 at 5:14 AM
    Re: Running ISA Server 2006 under Hyper-V
    Why three NICs? The comments seem to imply that two NICs would have meant registry hacking. I'd have thought it was possible to use two NICs, one on the outside network, with TCP/IP turned off as you detail, and one on the inside network with TCP/IP turned on (not strictly necessary, I guess, but makes sense for remote management), and then bridge the inside adapter on ISA, and the adapters for any other VMs to the inside adapter.

    So, ISA gets sole control over the external adapter and all the VMs and the host share the internal. That doesn't work?

  • Gravatar
    ThomasL Wednesday, October 15, 2008 at 2:54 PM
    Re: Running ISA Server 2006 under Hyper-V
    Why three NICs? The comments seem to imply that two NICs would have meant registry hacking. I'd have thought it was possible to use two NICs, one on the outside network, with TCP/IP turned off as you detail, and one on the inside network with TCP/IP turned on (not strictly necessary, I guess, but makes sense for remote management), and then bridge the inside adapter on ISA, and the adapters for any other VMs to the inside adapter.

    So, ISA gets sole control over the external adapter and all the VMs and the host share the internal. That doesn't work?

    • Gravatar
      Subodh Sunday, October 26, 2008 at 6:53 PM
      Re: Running ISA Server 2006 under Hyper-V
      @ThomasL

      Yes, Ideally it should work with just 2 NICs but it doesn't. You'd find that after about a day or two of normal operation, packets suspiciously start getting rejected at the ISA that originate and end at the same virtual network.

      This is not to say you cannot use just 2 NICs, you definitely can. Depending upon your configuration, you may not even need a dedicated NIC for access to the host machine.

  • Gravatar
    Anthony Mason Thursday, December 04, 2008 at 9:57 PM
    Re: Running ISA Server 2006 under Hyper-V
    Firstly, thanks for a really great post. This setup with ISA 2006, DC and Exchange in child partitions is exactly what I have been trying to achieve for two days!

    Firstly, is the overview diagram above showing the incorrect pNIC going to the router? Shouldn't it be 'Physical Gateway'? Your description seems to state so.

    I am a little confused though as to which vNICs connect to which vSwitches, pNICs etc. If I understand correctly you have:
    Parent partition
    vNIC1 > Virtual Internal > Physical Internal pNIC > pSwitch
    vNIC2 > Virtual Gateway > Physical Gateway pNIC > DSL modem
    vNIC3 > Virtual Public > Physical Public pNIC > pSwitch
    ISA child partition
    vNIC1 > Virtual Gateway > Physical Gateway pNIC > DSL modem
    vNIC2 > Virtual Public > Physical Public pNIC > pSwitch

    How about DC & Exchange, are they both unihomed and do they both use Virtual Internal or Virtual Public, ie:
    DC child partition
    vNIC1 > Virtual Public > Physical Public pNIC > pSwitch ??
    Exchange child partition
    vNIC1 > Virtual Public > Physical Public pNIC > pSwitch ??

    Also, I appreciate that in point 8 the binding on the physical NIC is to the vswitch rather than TCP/IP stack, but why is it that the bindings for the Virtual Gateway and Virtual Public are all unchecked, yet Virtual Internal are all checked and bound to TCP/IP as you'd expect? What does having no bindings like this mean?

    Apologies for the huge comment, this post has really helped me out, thank you.

    • Gravatar
      Subodh Friday, December 05, 2008 at 1:02 AM
      Re: Running ISA Server 2006 under Hyper-V
      @Anthony Mason
      Thanks for taking the time to write in ... Please let me know if the following answers/doesn't answer your question.

      The overview diagram is correct. The 'Physical gateway' is meant only for the private LAN side on the ISA machine. Hence also the reason why it is not connected to Virtual Internal Network from Hyper V itself (although as Thomas asked above, you can do that but in my experience ISA will get confused plenty of times and refuse to work occasionally).

      Gateway essentially is the default gateway for every computer on your internal network. This includes Virtual machines as well as physical machines. eg 192.168.1.1 etc ...

      To put things simply, the Host machine needs to stay isolated from all Physical connections both due to security as well as buggy Hyper-V/ISA combination reasons.
      If you were to put an ISA server in a physical machien of your own, you would make sure that the only interfaces in or out of that machine are the Physical Public and Physical gateway. The Physical Gateway would go to a switch whereas Physcial Public would go to your router and thence to the Internet.

      In case you're running it as a virtual machine, you essentially would try to do the same thing, except the 'Physical' is served by 'Virtual'. And the Host machine does not have access to Physical components (even though its easy to provide). To serve the connectivity between the host machine itself , we use a third NIC which acts as a bridge between the Internal Network via the switch) and the Virtual Network.

      Hope this helps.. If not, please feel free to comment. I'll also update with a Physical Network diagram in the post above so its clearer , what I described in this comment. Would that be helpful?

  • Gravatar
    Anthony Mason Friday, December 05, 2008 at 1:05 PM
    Re: Running ISA Server 2006 under Hyper-V
    Thanks for your response Subodh - the additional comments above do help with the ISA specific NICs, but I'm still not 100% clear on the Virtual Internal! Updating the diagram would be great, a list of all NICs with IP addresses & subnet masks would be very useful too! Thanks again, I really appreciate your help!

  • Gravatar
    Subodh Saturday, December 06, 2008 at 4:34 AM
    Re: Running ISA Server 2006 under Hyper-V
    Anthony, Just updated the article above with some more information. Hope this helps.

  • Gravatar
    Anthony Mason Monday, December 08, 2008 at 12:39 PM
    Re: Running ISA Server 2006 under Hyper-V
    Subodh, many thanks for taking the time to add the extra details. - that was very kind of you. On the strength of this post, I've just ordered a new Dell PowerEdge server and will be putting this exact setup into production in the coming weeks! Keep the great blog posts coming! A.

  • Gravatar
    Anthony Mason Friday, December 19, 2008 at 1:37 PM
    Re: Running ISA Server 2006 under Hyper-V
    Hi Subodh, I thought I'd feed back my slightly different version of your setup. All appears to be working perfectly! happy

    HOST
    -Dell PowerEdge 2900III, 8GB RAM, 2X146GB SAS RAID1, 5X 146GB SAS RAID5
    -Windows Server 2008 EE x64
    -3 Physical NICs
    --Physical Public (bound to MS Virtual Network Switch Protocol only)
    --Physical Internal (bound to MS Virtual Network Switch Protocol only)
    --Physical Gateway (bound to MS Virtual Network Switch Protocol only)
    -3 Virtual Networks
    --Virtual Public (no bindings)
    --Virtual Internal (bound to everything but MS Virtual Network Switch Protocol)
    --Virtual Gateway (no bindings)
    -1 physical switch connected to Physical Gateway and Physical Internal on HOST
    -Third physical NIC (Physical Public) connected to Netgear DG834GT.

    Netgear DG834GT ADSL Modem/Wireless AP/Router/NAT/Firewall
    -In Router mode (not Bridge mode, I could not get this to work after major effort!)
    -Static WAN IP: w.x.232.48 /32, PPPoA
    -LAN IP: 192.168.1.1 /24
    -Static Route: Destination IP 192.168.0.1, SM:255.255.255.0, Gateway:192.168.1.2

    ISA1 VM, Win2K3 Std (not member of domain)
    -Edge Firewall template
    -Virtual Public adapter - IP:192.168.1.2, SM:255.255.255.0, DG:192.168.1.1, DNS1:192.168.1.1, DNS2:194.74.65.69
    -Virtual Gateway adapter - IP:192.168.0.1, SM:255.255.255.0 DG:blank, DNS1:blank, DNS2:blank

    DC1 VM, Win2k3 Std
    -Virtual Internal adapter - IP:192.168.0.2, SM:255.255.255.0, DG:192.168.0.1, DNS1:127.0.0.1, DNS2:blank
    -xyz.local domain, with forwarder to ISP DNS, DHCP

    EXCHANGE1 VM, Win2k3 Std
    -Virtual Internal adapter - IP:192.168.0.3, SM:255.255.255.0, DG:192.168.0.1, DNS1:192.168.0.2, DNS2:blank

    WSS1 VM, Win2k3 Std
    -Virtual Internal adapter - IP:192.168.0.4, SM:255.255.255.0, DG:192.168.0.1, DNS1:192.168.0.2, DNS2:blank

    XP1 VM, XP Pro SP3
    -Virtual Internal adapter - IP & DNS Obtain automatically.

    Thanks again for your help in setting

  • Gravatar
    Todd Wednesday, January 21, 2009 at 8:23 AM
    Re: Running ISA Server 2006 under Hyper-V
    Thanks for a great article.

    is there any tricks to installing ISA?

    ive installed on a 32 bit 2008 server vm & havent got any option of cennecting to the local isa box in the isa console. it only give me an option of connecting to a different server.

    thanks

  • Gravatar
    Todd Saturday, January 24, 2009 at 5:24 AM
    Re: Running ISA Server 2006 under Hyper-V
    Make sure you dont try installing ISA on a 2008 Server box !!!

  • Gravatar
    Tarek Majdalani Saturday, January 24, 2009 at 9:30 AM
    Re: Running ISA Server 2006 under Hyper-V
    Hi, you said : The DNS entries for the ISA server are : External (public) interface points to ISP DNS server; Internal(private) interface points to internal DNS server.

    Thats absolutely wrong, never put any External DNS Entry on any of the ISA Server Network Cards. It should only have a reference to the Internal DNS Server which shoiuld then forward external requests to the ISP DNS Servers.

    • Gravatar
      Subodh Thursday, January 29, 2009 at 7:16 AM
      Re: Running ISA Server 2006 under Hyper-V
      @Todd
      There is nothing wrong with installing ISA 2006 on 2008 Server VM however, I would always recommend installing ISA 2006 on a Windows Server 2003 VM since there are a few compatibility issues that one needs to sort out and unless you are a dedicated Systems Admin, the ROI is not worth it.

      @Tarek
      Could you please explain why it is "absolutely wrong" to put external DNS entry on external ISA network? Theory is one thing, practically you would not want the ISA server to lose connectivity (DNS wise) if the internal DNS server is down anyway. Having the ISA server member of the domain allows you to have the best of both worlds and in no way compromizes security. In my experience, ISA pointing to internal DNS server ( which itself is virtualized) is asking for trouble.

  • Gravatar
    jonas Thursday, April 16, 2009 at 11:50 AM
    Re: Running ISA Server 2006 under Hyper-V
    The key here as I understand it is to keep the host computer unaware of the public connection, and only forward that connection to the ISA VM.

    Why can't I just in the host OS uncheck everything (as you did) for the virtual NIC representing the public connection and be done? That would totally hide that connection to the host OS, but I could still use it in the ISA VM, right? I stil don't understand why you need the three NICs confused

  
Locations of visitors to this page Clicky Web Analytics 

Subodh's Blog Rating

 

DISCLAIMER

The opinion expressed
on this page 
is strictly that
of the page author
who has a
habit of animating
day-dreaming
and
fictionalizing
out of thin air.
 

The contents of this page
have not been
reviewed 
nor
approved
by 
Yahoo!

 Follow this blog
  
 Tag Cloud
  
Archives
 

Top 5 Posts of Last year
Copyright © 1995-2009 Subodh Shakya. All rights reserved.{Powered by SpeedBlog}