Semiproductive Semiconductor

One of my clients is setting up two new VPNs — one is a connection between two office locations, and the other is the ability for users to dial in from home.

So, in either of the offices, we don’t care about reaching the single-PC VPN machines, but those machines do need to be able to reach computers at both offices. This article covers the routing situation for single-PC clients.

For hardware, we chose the SonicWall TZ100 as our appliances; one in each office. They’re pretty nice — they do VPNs,firewall, Antivirus, Spam, automatic failover/load balancing. They also provide L2TP Services, which work pretty well with the Windows Native VPN client.

We will have a couple of subnets, and a couple of VPN users; that’s a somewhat more complicated IP situation than the default situations provide. Our active subnets are:
192.168.1.x (main office),
192.168.2.x (remote office), and
192.168.4.x (VPN DHCP pool).

So, on a client PC, we want to route any traffic intended for those subnets to go through the VPN, any other network traffic we want to go out through the regular internet connection.

I was quite surprised to find out how difficult this is to pull off!

I finally settled on using WSCRIPT to accomplish that. You have to create the VPN connection item yourself, but then you can run a script connect the VPN, query the system for the IP address and set the routing.

Of course, you first need to create a stock Windows VPN connection. Then you have to set the Windows VPN item to disable the “make this gateway the default”. Then, you’ll need to use a bunch of ROUTE ADD commands to manually add the routes to the routing table.

However, the catch to that is that you need the IP address that the VPN provides, so that you can tell the computer which IP address to use as the route. There is no elegant way to get this information. There are a few WMI classes that look promising: Win32_NetworkAdapterConfiguration and Win32_NetworkAdapter. However, they only return the hard-coded IP address (or 0.0.0.0 if it’s DHCP) of the adapter, so that’s not useful.

My next thought was to fetch that information out of the registry. When the connection is made, that IP address is stored in the Registry, but only in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters\Interfaces\{someGUID}. That key is identified with a GUID; you’d think that GUID would match up with something else in the system, but I sure couldn’t find it. I don’t mind using GUIDs in scripts, but trying to write a script that you can give to an end-user that requires them to locate and enter a GUID is a bad idea.

SO, there’s a lot of non-attractive options out there. I did find a piece of code on the web that ran IPConfig and piped it into an array, parsing it to pull out the IP addresses returned for any adapters connected to the machine. Piping the output of a utility into a string and parsing it is error-prone — who knows if Windows 8 will change out IPConfig works. But I don’t think there were any other options. I modified it somewhat to pull the IP address for a specified adapter:

That was the hard part. Now for the rest of the code.

As you can see, it calls the “rasdial” command, which is the command that loads and runs a VPN connection. (It sure would be nice if such a command could be modified with a parameter to return the IP address, but no such luck.) Then once the connection is established, we fetch the IP address, clear the existing route table entries, and add the new ones.

And that’s it. Feel free to use this; hopefully it’ll save you a little time somewhere.

If anyone feels like adding, or pointing people to, a script to create the VPN connection in the first place, I’d be delighted to see it.

Full Source:

So one of my clients does a lot of data analysis on large, wide data sets. (Think 4000 columns and 10 million rows.) One of the thornier problems we face in processing is a variant on the Nearest-Neighbor calculation. The columns we use are all various demographic indicators of a set of people, and we generate the pythagorean distance from each person in the set to each other person. The Pythagorean distance is merely a giant hypoteneuse — for a simple triangle, H = sqrt(a² + b²). Well, for a multi-dimensional calculation (purely theoretical), H = sqrt(a² + b² + ….. + n²). Then, suppose you have John Q Public, and you know 100 different tidbits about him (age, gender, level of education, income level, marital status, kids, own/rent.)

If you can find a way to convert those attributes into meaningful numbers, you can then generate a hypoteneuse as follows:

let:

• A be the percentile of Age;
• B be percentile of years of education;
• C be percentile of salary; and
• D be percentile of how many children one has.

JQP is the index in the vector for John Q Public.  x is the index for any given other person)

	H[JQP][x] = sqrt((A[JQP] - A[x])² + (B[JQP] - B[x])² + (C[JQP] - C[x])² + (D[JQP] - D[x])²)

As you can imagine, this turns into a lot of computation – number of total row comparisons is on the order of, N², and the number of column comparisons is linear. So 10 million row comparisons comes to 50 trillion items at 4000 subtractions, squares, additions, and a square root per row.

Now, some of the details of generating the distance, and its value to the real-world problems that are being approached is debatable, but I’m just the coder, so that’s not my problem. Besides, those are details that can be tweaked once we have a number-crunching mechanism in place.

Fortunately, this also lends itself very well to parallelization. Each computation can be done completely independent of each other. You may see optimizations in grouping several thousand nearby row comparisons in a chunk of memory, but in the abstract, you could farm out any individual row to a girl in Borneo with a calculator and wait for her to finish. (This solution is not recommended as it does not scale well.)

Detailed optimizations and file-management factors aside, this is an excellent case for experimenting with GPUs as computational platforms. ATI and nVidia are major GPU platforms, and Intel claims to be working on their own GPU. We were looking at CUDA and DirectCompute, but chose OpenCL for our proof-of-concept. OpenCL is a nice compromise between CUDA’s closed architecture and DirectCompute’s overly-generic approach. OpenCL is a common layer for both ATI and nVidia, and claims to be working on a broad array of other hardware architectures.  In my experience, it’s very similar to the CUDA code; some of the keywords and compilation processes are a little different, but it’s the same general idea. I’m eager to see what, if any, headway they’ll be making on other architectures like the Cell processor and the like.

There are copious documents available (and some particularly good ones from nVidia) to describe the architecture of a GPU, but here’s the basics: You have a large chunk of general-purpose RAM (fast by PC standards, but very slow compared to the math units on the card), a handful of Logic Processors (LU) that coordinate threads, move data around, and other boring stuff, and a large number (16 per LU) of math units. These math units have 64K of ram each, run in complete parallel to other math units, and are capable of running multiple threads on each, so each task doesn’t really have 64K of ram available — that’s shared among some other number of threads running on the math unit. The LU coordinates these tasks and instructs the math units what to work on and when to do them. (Including giving them other work to do while they’re waiting on results from global memory.) There’s also a small amount (64K) of fast shared memory for each LU, so each of the math units can share some data with one another. The individual tasks that are run are called Kernels.  A Kernel is an instance of a task to perform; the idea is that there will be a lot of Kernels run, some on each math unit and managed by the LU.

For example, suppose you have 100 numbers for which you need to generate their cube root. You would create an array of 100 source numbers and a blank array for 100 result numbers. You have 1 LU and 16 math units. The LU will say “ok, math unit A, you take kernel instances 0, 16, 32, 48, 64, 80, and 96. Math unit B, you take 1, 17, 33, 49, 65, 81, and 97.” And so on until all 100 kernels have been instantiated and assigned to a math unit. Each math unit will do a few cycles of preparation, then for kernel 0, it’ll have to do the original fetch of source number 0 from global memory. It issues the fetch instruction, and now has a few hundred cycles on its hands before getting a result. (Each of the math units will be in the same situation.) So the LU will instruct them to get started on kernels 16, 17, 18, and so on. They’ll issue the fetch for that and so on, until either they’re out of new jobs to start or get their results back for kernel 0. Once they have their source numbers, they perform the math on them. When the math is done, they send the numbers back out to their destination and address the next thread that’s ready to have work done on it.

Obviously, this can get much, much more complicated, but that’s the general idea. Refer to the nVidia documentation if you want to know more.

So we wanted to build a somewhat-generalizable platform for experimentation that would let us leverage what we wrote in the future. So the first task was building the boring stuff: datafile load/save and error-reporting mechanisms. OpenCL is in C, so I had to refamiliarize myself with mallocs and the like. I just used tab-delimited files to store a series of ASCII integers between 0 and 99 as the input file, and a series of ASCII floats for the output.

The output format is going to be a triangle – there’s no need to compare row 1 to row 3 AND row 3 to row 1, so the output is (N * (N-1))/2 cells arranged in N-1 rows.

Once the I/O stuff is handled, the next step is to allocate large buffers in memory (ultimately, production-level scalability concerns will involve breaking the data into chunks that can fit into RAM on both the host computer and the GPU) for both the source and the result data, and also creating the “global memory” buffer on the GPU for the same purpose. Next, I created a general plan of attack for the Kernel. Each math unit has 16K (16384) 32-bit floating-point registers; that means for a 4000-column dataset, you can hold 4 threads’ worth of working data if you’re careful with it. (See above about multiple threads for math units)

In this case, we’ll be going from 1 to N, comparing each row’s predecessor to all the rows after it. So, here’s the code that creates the kernels and sets up their parameters:

you can only have 512 threads in a work group. More than that, and you’ll need to create another work group. So, Global_work_size >= local_work_size, and must be a multiple of local_work_size. If there’s a remainder, we need to create the threads for all the remainders and put a “don’t do anything” in the kernel for the instances that are not needed.

Note the comment about work groups:
you can only have 512 threads in a work group. More than that, and you’ll need to create another work group. So, Global_work_size >= local_work_size, and must be a multiple of local_work_size. If there’s a remainder, we need to create the threads
for all the remainders and put a “don’t do anything” in the kernel for the instances that are not needed.

Similarly, there’s a performance tweak in here about “strides” — the GPU access data a little faster if all the fetches line up to some multiple of the number of math units in a thread group. So it’s slightly faster to make sure the width of your rows is some multiple of the number of math units. This means that the width of the row might not equal the number of cells. That’s why there are separate parameters for RowWidth and NumCols.

The other parameters are pretty straightforward — Base row, which is the row against which all the kernels are comparing their rows, the number of rows in the set, number of kernels in the iteration, and pointers to the beginning of the input and output arrays.

All the calls to the GPU are asynchronous, but you can fake synchronicity by making the calls blocking.

Now let’s take a look at the Kernel code. A rather odd (IMO) feature of OpenCL is that the kernel is compiled at runtime from a separate source file. That wouldn’t be my preference, but it’s not a very big deal, since the kernels aren’t very big, and the whole point of the process is to cut down on stuff that would take minutes or hours to run. So it’s acceptable, but still rather strange.

the __global prefix means that it’s global memory on the video card.

NumCols doesn’t necessarily equal RowWidth — RowWidth is rounded up to a size that optimizes memory fetch — usually a multiple of 32 or something.  NumCols is the number of columns that actually contains data.

What I’m doing is having only thread #0 move a copy of the base row to shared memory. This shared memory is very fast and is accessible from all the kernels, so we might as well use it. It seems rather strange that the only way to do this is have one Kernel do the move and have the rest sit around, but that seems to be the case. We could streamline this somewhat by having each thread move a few bytes, if that helps.

This barrier call forces the Logic Unit to wait till all the kernels have reached this point (obviously, thread 0 will be the critical path on this one) before any of them progresses beyond this point. Since all the kernels will be depending on the data move above, we need them to wait.

As I stated before, the output array will not be rectangular, so we have to calculate the proper place to put the result. We could be
inefficient in our use of memory space, but we’d run into problems with that eventually, so we might as well do it right.

…And that’s it. The performance of this was not very good, frankly; I was able to get about a 15% increase over a CPU-based version of the same calculation with a 1000×1000 input array. The bulk of the computation gains were eaten up by the overhead of compiling the kernel and moving the data to/from the GPU. Unfortunately, when I tried to scale it up, I ran into memory problems that were outside the scope of the proof-of-concept project.

Further efforts on this will focus on scalability issues and more efficient data-management approaches.

Well, i’m gonna give it a shot.  We’ll see how it works out.