New Home Server. Help Needed.......

Posted by: Cris

New Home Server. Help Needed....... - 29/03/2006 16:09

After many many many years I have decided to swap my ISP from BT to Zen in order to get myself a range of fixed IP's. The end goal is access from the web of several devices I have on the network (cctv, tivo, future home automation etc...).

This all works to a point at the moment, I host the different services on different ports and use the features in my DSL router to point things in the right direction. What I would like is one single secure web gateway/interface to all these systems with a secure login etc...

I would like to keep general internet access seperate from this system and behind a firewall of some kind. I have a spare PC and the current cctv server ready for use, but I am not finding much that can do what I want and I am way out of my depth, so I thought someone here may like to point me in the right direction

Cheers

Cris.
Posted by: tman

Re: New Home Server. Help Needed....... - 29/03/2006 16:18

You can run Apache on your gateway and get it to proxy stuff into your internal network. Turn on passwords and it should be reasonably secure as well.

If you don't want it on the gateway then port forward from the firewall to your webserver box instead.
Posted by: mlord

Re: New Home Server. Help Needed....... - 29/03/2006 16:24

If you really want to use a range of external IP addresses (rather than DNAT on a firewall), then the thing you need in front of them all is a "bridging firewall" (or "transparent" firewall).

Linux can do this, of course.

Cheers
Posted by: wfaulk

Re: New Home Server. Help Needed....... - 29/03/2006 16:28

Quote:
You can run Apache on your gateway and get it to proxy stuff into your internal network.

In addition to which, it not only doesn't need separate IP addresses, it won't really work with separate IP addresses.
Posted by: Cris

Re: New Home Server. Help Needed....... - 29/03/2006 16:45

So what would be the best option to use my slimserver from both side of the network?

Are there any good "dummy" type websites for this stuff?

Cheers

Cris.
Posted by: wfaulk

Re: New Home Server. Help Needed....... - 29/03/2006 17:15

I'd set it up with Apache's "reverse proxy" feature. Just google on that and see what comes up. I assume that it'll work on a non-Unix Apache, but I don't know for sure.
Posted by: Cris

Re: New Home Server. Help Needed....... - 10/04/2006 20:58

So, after looking into the whole thing I have ended up running ClarkConnect. It's my first outing onto a Linux based system, and I seem to be getting on with it fine. I haven't got to grips with the whole reverse proxy thing yet, but I get the idea.

It's all running off a 1GHz C3 Mini-ITX board in one of these! I didn't pay that price for it, they are really nice people, I went to see them and they did me a really good deal. I also splashed out and got 2 new 250Gb WD RE from Scan. I am so happy with the format, I think I am going to shift the CCTV server into one too

Thanks for the pointers, I am learning lots of new things here, I even found myself picking up books on PHP and MySQL at the bookshop over the weekend

Cheers

Cris.
Posted by: StigOE

Re: New Home Server. Help Needed....... - 11/04/2006 04:40

I have been running ClarkConnect for several years and I'm also very happy with it. I'm also running my mailserver and webserver off it.

Stig
Posted by: andy

Re: New Home Server. Help Needed....... - 13/04/2006 17:00

Quote:

It's all running off a 1GHz C3 Mini-ITX board in one of these! I didn't pay that price for it, they are really nice people, I went to see them and they did me a really good deal. I also splashed out and got 2 new 250Gb WD RE from Scan. I am so happy with the format, I think I am going to shift the CCTV server into one too



Thanks to your post I finally spurred myself into action and bought the parts for my silent server (to replace the two big noisy and hot servers that currently live in my garage).

I was going to buy my T2e chassis from mini-itx.com, which would have cost me £150. After your post I bought it direct from Tranquil for £65 (via a "best offer" bid) ! Thanks.

I also got a PSU and EPIA SP13000 from them at the same time, with only £10 postage for the lot.

I've just put it together, along with 2 250GB Seagate NL35 drives (or at least the drives will get plugged in when I pick up a couple of SATA cables). Getting the old heat sinks off of the CPU and Northbridge was much easier than I was expecting.

Once I've soak tested it I'll have the joy of getting Debian + SATA + software RAID1 (on the boot drive) up and running.

I'll probably get another 250GB drive and fanless external USB2 box to act as the spare disk in my array.

P.S. does any one know if the memory limit for the EPIA SP13000 really is 1GB. It has a single slot and everywhere I read that it can take a maximum of a 1GB stick of RAM, but I have come across cases before where motherboards could in fact support higher density RAM, even when all the docs said they couldn't. A case in point is my old NEC laptop, all the docs say it can only handle 128MB RAM (limiting it to 256MB) whereas in reality it was happy using 256MB RAM sticks (taking it to 512MB).

Unfortnately (or maybe fortunately) I don't have a 2GB stick of RAM to try it with...
Posted by: Cris

Re: New Home Server. Help Needed....... - 13/04/2006 17:53

Glad to be of help

They are great cases for the price, I'm in Manchester quite a bit these days, so I went down to their unit to pick them up. Very nice people, willing to offer loads of advice, and gave me a demo of just about every single product they offer.

Check out the cheesey photo of my under the stairs rack...



The 4U server above the new mini-atx, is very loud, comes out at about 55dB on my sound meter, so I think the swap out will happen sonner rather than later.

Cheers

Cris.
Posted by: andym

Re: New Home Server. Help Needed....... - 13/04/2006 19:59

Quote:
Check out the cheesey photo of my under the stairs rack...


Is that a BT patchbay I see there?
Posted by: matthew_k

Re: New Home Server. Help Needed....... - 13/04/2006 20:35

That's a nice little home rack you've got there, is that off the shelf or did you make it yourself?

Matthew
Posted by: mlord

Re: New Home Server. Help Needed....... - 14/04/2006 00:48

That's a great looking rack, Cris(sie)

Definitely custom. MDF?

Those massive power plugs really add to the effect -- much more impressive than the skinny wee ones we have over this side!

Cheers
Posted by: Cris

Re: New Home Server. Help Needed....... - 14/04/2006 07:02

Quote:
Is that a BT patchbay I see there?


Umm, yea, I found that in a skip

Quote:
Definitely custom. MDF?


It's sort of custom, I got it from a company called Wizzard, they make rackmounts to go in music and TV studios, but they seem to have gone bust as their website is down? It's a beech veneer finish, so a little over the top for my needs, I think I would make the next one myself out of MDF.

One problem I do have is cooling the room, it doesn't get very hot, but hot enough, it's a sealed room and can get upto 32 if I leave the door shut for a week or two, does anyone have any ideas about cooling a sealed room like this without noisy fans etc... Unfortunatly as it's under the stairs and in the middle of the house there is no access to an outside wall. I was thinking of some kind of temp sensor that could control a small fan, and report to a webpage of course! Something a little like the WeatherDuck?

Or, am I going a little over the top for a home setup

Cheers

Cris.
Posted by: sn00p

Re: New Home Server. Help Needed....... - 14/04/2006 07:33

Yeah, that looks a little over the top.

I'm sure theres more than one of us on here who could knock you something up with a NTC/PTC/dallas one wire sensor + microcontroller (dare I say it PIC/AVR) that would dump the temperature to the serial port.

Actually, I bet there's hundreds of PIC projects on the web that do what you want!
Posted by: andy

Re: New Home Server. Help Needed....... - 14/04/2006 07:59

When I wanted temperature monitoring I got this kit, complete with pre-programmed PIC for about $25. It is still plugged into my server and working well.

Thanks to my T2e purchase I shouldn't need to worry about monitoring the temperature in my garage for much longer.
Posted by: boxer

Re: New Home Server. Help Needed....... - 14/04/2006 08:09

Quote:
cooling a sealed room

What you have there is a useful means of providing background heat for the rest of the house, without using valuable energy resources, I'm sure Tahir would applaud! All you have to do is install a couple of small ducts and a fan, to take it to somewhere useful, with perhaps an arrangement to switch it to outside in warm weather. That is, unless, of course, you can think of a way of running the pipes through your hot water tank.
Maybe I'll get a set up like that for under the stairs, if only I could think what to do with it!
Posted by: wfaulk

Re: New Home Server. Help Needed....... - 14/04/2006 17:08

Your computer probably has some sort of in-case ambient temperature sensor. It won't be the same as the room's ambient temperature, but I bet it's pretty easy to assume from that.
Posted by: Cris

Re: New Home Server. Help Needed....... - 14/04/2006 21:53

If you look close at the 4U rackmount case it has 2 temperature displays on the front, one is on the HD's and one is the room temp, the sensors are on leads so they can be positioned, so I know there is a heat problem from looking at them.

What I want is a solution that will allow me to monitor the temps on a webpage, and automatically activate some form of cooling system when needed, without me having to go open the door for a bit

It's not major problem at the moment, but if I do move to a server system that has no active cooling at all it could become one I think. For now I guess I will just keep on opening the door every few days and letting the heat escape into the house.

Cheers

Cris.
Posted by: andy

Re: New Home Server. Help Needed....... - 15/04/2006 04:58

Quote:

What I want is a solution that will allow me to monitor the temps on a webpage, and automatically activate some form of cooling system when needed, without me having to go open the door for a bit



In that case you want something like this. With this RS232 board you can read the temperature and control devices via the relays it provides.

The serial commands are all simple, it could all be implemented with some simple Linux shell script and a crontab job.

It isn't available as a kit, only a ready built board, but it is still only £30.
Posted by: Cris

Re: New Home Server. Help Needed....... - 15/04/2006 06:16

Thank looks just the ticket! Is it easy to get the result displayed on a webpage using those methods? That is one area I don't have a clue in.

Cheers

Cris.
Posted by: andy

Re: New Home Server. Help Needed....... - 15/04/2006 07:34

Relatively easy, yes. For example, I display the temperature of my garage here. You can see all the other crap that I monitor here.

To do this I use a couple of shell scripts and mrtg.

N.B. anyone who is any good at Unix shell scripts, sed and the like, cover your eyes now. My Unix commandline knowledge is just barely enough to get by with stuff like this and I know it is nasty...

The first shell script is called "tempread" and looks like this:

Code:
#!/bin/bash

echo `head -n 3 /dev/ttyS0 | tail -n 1`


When you start reading data from the serial port (/dev/ttyS0) you get the following output:

Code:

R V1.0 2001-10-13 11:56:11 C



1 0012.68



1 0012.68



1 0012.75



... (it keeps outputing the temperature until you stop reading


So my shell script uses "head" to grab the first three lines of the output of the serial port. It then uses "tail" to grab the last line of the three.

The "tick" marks, i.e. the ` characters, tell bash to run the command inside and return the text generated.

So the output of tempread looks like:

Code:

1 0012.68


The next shell script is "tempmrtg":

Code:

#!/bin/bash

echo "`/usr/local/bin/tempread |cut -c 5-9` * 100" | bc | sed s/.00$//

echo "0"

echo "0"

echo "0"


This is the script that generates the output for mrtg. Because mrtg is really designed for monitoring router throughput usage it only deals with integers. It expects four values from any scripts it calls. The first is the throughtput in, second throughput out and the last two I forget.

Because I want better than whole number resolution I multiply my temperature by 100. So if the temperature is 12.68 I end up returning 1268 to mrtg.

The first line of "tempmrtg" calls "tempread", extras the temperature digits using cut, combines the temperature with "* 100" and passes that to bc (and command line calculator). It then uses a regex and sed to strip the ".00" off of the end of the resulting value.

The last thing is an entry in the mrtg config file that tell it to call "tempmrtg" to get the values:

Code:

Target[temp]: `/usr/local/bin/tempmrtg`

MaxBytes[temp]: 4000

YTicsFactor[temp]: 0.01

Factor[temp]: 0.01

Options[temp]: gauge, nopercent, noo, transparent

XSize[temp]: 600

YSize[temp]: 150

YLegend[temp]: Temp

ShortLegend[temp]: C

LegendO[temp]:  Nowt:

LegendI[temp]:  Temp C:

Title[temp]: Server room temp

PageTop[temp]: <H1>Server room Temperature

 </H1>

  <TABLE>

    <TR><TD>System:</TD><TD>Server room temp</TD></TR>

  </TABLE>


Amongst other things this tell mrtg that:
- it should run "tempmrtg" to get the values for this graph
- that it should divide all the values by 100
- that the temperature will never be more than 40 degrees
- that there is only one value to graph ("noo" tells it that)

Now that I have written all that down it seems a little more complex than I remembered

There are tools that I probably better than mrtg for this stuff now. I use mrtg because that is what I know. The most likely canditate would be rrdtool.
Posted by: andy

Re: New Home Server. Help Needed....... - 15/04/2006 07:39

The bbs is double spacing all my code blocks and not using a sensible font, make it stop...
Posted by: sn00p

Re: New Home Server. Help Needed....... - 15/04/2006 08:11

I did the same, best offer of £50 accepted!
Posted by: sein

Re: New Home Server. Help Needed....... - 15/04/2006 08:28

I am using Cacti to graph useful information about my server at home.

Fan speed control is actually done in hardware by my Motherboard (an Abit something with a uGuru chip). Temperature monitoring should in theory be a piece of cake in Linux with an lm_sensors compatible motherboard.
Posted by: julf

Re: New Home Server. Help Needed....... - 15/04/2006 10:07

Quote:
N.B. anyone who is any good at Unix shell scripts, sed and the like, cover your eyes now.


It was pretty hard to read with my eyes covered , but...

How about replacing the two first scripts with this:

Code:

sed -n -e '3s/^....\(..\)\.\(..\)/\1\2\n0\n0\n0/p'


Given your input data, it produces:
Code:

1268
0
0
0
Posted by: andy

Re: New Home Server. Help Needed....... - 15/04/2006 10:11

For those who don't know, Cacti is a front end to the RRDTOOL I mentioned earlier. RRDTOOL was developed by the same guys who did MRTG, when the decided that MRTG's design was too limited for general graphing.

I'll probably give Cacti a go when I get my new server up and running fully. I'll still need my shell scripts however to gather data from things like my temperature board.
Posted by: andy

Re: New Home Server. Help Needed....... - 15/04/2006 10:19

Thanks Julf !

The problem is that I use sed about once a year, so I never actually completely grasp how to do stuff like that. It can be so frustrating knowing there is a better way to do things but not being able to work out how to do it...

While we are talking about sed, what would be the "right" way to extract both the "DS Margin" and "DS Line Attenuation" values from this input:

Code:

        DS Line Attenuation:    40      DS Margin:              8


So far I have:

Code:

cut -d: -f3 | sed -e "s/ *//"


Which gets me the margin value, but I am stuggling to separate out the attenuation value.
Posted by: andy

Re: New Home Server. Help Needed....... - 15/04/2006 10:27

Hmmm, the \n doesn't give me a newline, it just outputs "n". Perhaps it is because I have a very old copy of sed (this box is still on RedHat 7.1).
Posted by: mlord

Re: New Home Server. Help Needed....... - 15/04/2006 10:27

Quote:
DS Line Attenuation: 40 DS Margin: 8


awk '{print $4 " " $7}'
Posted by: mlord

Re: New Home Server. Help Needed....... - 15/04/2006 10:32

You first script/sed problem is too spread out for me to grok here. Can you summarize it in one post: inputs, and expected output? I believe it can be much simpler than even Julf's one-liner.

Cheers
Posted by: mlord

Re: New Home Server. Help Needed....... - 15/04/2006 10:42

Quote:

R V1.0 2001-10-13 11:56:11 C
1 0012.68
1 0012.68
1 0012.75

... (it keeps outputing the temperature until you stop reading

So my shell script uses "head" to grab the first three lines of the output of the serial port. It then uses "tail" to grab the last line of the three.

...
Quote:

1268
0
0
0



Okay, so not overly simplified, but this version handles temperatures that have more/fewer leading zeros.

awk '/^1 /{ if (++i == 2) {print gensub("^0*","","g",gensub("[.]","",1,$2)) "\n0\n0\n0";exit}}'

In multi-line format, it's easier to read (or would be, if the BBS could use a proper fixed font for code display!):Code:

awk '

    /^1 /{

        if (++i == 2) {

            tmp = gensub("[.]","",1,$2)

            gsub("^0*","",tmp)

            print tmp "\n0\n0\n0"

            exit

        }

    }'
Posted by: mlord

Re: New Home Server. Help Needed....... - 15/04/2006 10:59

Quote:
sed -n -e '3s/^....\(..\)\.\(..\)/\1\2\n0\n0\n0/p'


Here's a more portable version, that also does not assume a fixed number of leading zeros :

sed -n -e '3s/^[^ ] *0*\([^.]*\)[.]/\1/p' -e'3s/.*/0/p' -e3p -e3p
Posted by: julf

Re: New Home Server. Help Needed....... - 15/04/2006 11:33

Indeed better, I just tried to keep it minimalistic, as the original solution was based on a fixed column position.
Posted by: julf

Re: New Home Server. Help Needed....... - 15/04/2006 11:34

Quote:
Hmmm, the \n doesn't give me a newline, it just outputs "n". Perhaps it is because I have a very old copy of sed (this box is still on RedHat 7.1).


Indeed sounds like an old version of sed. Marks solution deals with that issue anyway.
Posted by: julf

Re: New Home Server. Help Needed....... - 15/04/2006 11:37

Quote:
awk '{print $4 " " $7}'


That what what I immediately thought of as well.
I would just use awk '{ print $4, $7 }' for clarity.

Awk must be the most under-utilized wonder tool ever.
Posted by: Cris

Re: New Home Server. Help Needed....... - 15/04/2006 14:54

If I just wanted to display the current temp on a "current server status" type page, how would I do it? I want a summary page that lists all the usefull stuff, dsl line rate, room/system temp, webcams etc...

As an expasion of this system I would like to monitor my home central heating system, surprisingly there seems to be very little about this on the web (that I can find), a thermostat with a web interface would be perfect

Cheers

Cris.
Posted by: andym

Re: New Home Server. Help Needed....... - 15/04/2006 15:11

You probably just want something like this....

http://www.eix.co.uk/Ethernet/SitePlayer/Welcome.htm

Think I've mentioned it before....
Posted by: sn00p

Re: New Home Server. Help Needed....... - 15/04/2006 15:39

Yeah.

We're using a Microchip ENC28J60 (10Mbit MAC+PHY) mainly because we're pin bound on a project and this thing uses SPI, but it requires only a couple of caps, resistors and a RJ45 (integrated magnetics).

What's more, if you were to pick a suitable PIC, microchip have a TCP stack that you can use straight up.

The ehternet chip is available in a 28 pin DIP from farnell at around £5 a pop.
Posted by: Cris

Re: New Home Server. Help Needed....... - 15/04/2006 19:52

Quote:
You probably just want something like this....


Yes I do, it looks perfect for what I need. Do you mind if I bend your ear when I get stuck (gunna happen!)

Cheers

Cris.
Posted by: andym

Re: New Home Server. Help Needed....... - 15/04/2006 19:59

Yup, sounds like fun!
Posted by: andym

Re: New Home Server. Help Needed....... - 15/04/2006 20:11

Looks like my bid of £55 was sucessful too! Another PC to add to the collection.... Damn you Matthews!
Posted by: Cris

Re: New Home Server. Help Needed....... - 15/04/2006 20:37

Go and pick it up from him, drop Amy off at the Trafford centre on the way Tell him I sent you, maybe I'll get some really good deals (to share) next time I go? I want a 3rd now, to go under the (future) HDTV.

Cheers

Cris.
Posted by: mlord

Re: New Home Server. Help Needed....... - 15/04/2006 21:30

Quote:
Awk must be the most under-utilized wonder tool ever.


Yes (except for here, of course!).

I just see so much stuff being done with SQL databases, PHP, and Perl, when it could all have been so much simpler as basic ASCII flat files with an awk script or two.

Cheers!
Posted by: andym

Re: New Home Server. Help Needed....... - 16/04/2006 06:38

Couldn't find their address anywhere and I've already paid for it to be delivered. I'll drop them an email and see if they'll let me pick it up.
Posted by: julf

Re: New Home Server. Help Needed....... - 16/04/2006 11:49

Quote:
I just see so much stuff being done with SQL databases, PHP, and Perl, when it could all have been so much simpler as basic ASCII flat files with an awk script or two.


Indeed. Back in my bad old days of teaching UNIX training courses in the 80's, I usually got people's jaws to drop when I showed some simple examples of the power of awk. Too bad nothing has changed in the last 25 years. (OK, they would have used something even more horrible than PHP and Perl...)

Cheers!
Posted by: andy

Re: New Home Server. Help Needed....... - 18/04/2006 14:21

I've got my server completely up and running now, I've learnt a few things about Linux software RAID along the way - like if you install a new kernel while an array is degraded it will always be degraded on reboot, until you re-run mkinitrd.

I have one problem though. It looks like the EPIA SP 13000 refuses to boot if it is hot. It will soft reboot quite happily, a "reboot" from the Linux command line works perfectly.

If however I use the power switch on the box to power it off and back on or yank the power it refuses to boot. It powers the main board (the Ethernet link light comes on) but doesn't turn on the VGA output or spin up the drives.

If I leave it a few minutes and try again it boots ok.

I think it was doing this before I replaced the fansink with the heat pipe, though I only booted it a couple of times before removing the fansink.

Anyone else seen anything like this ?
Posted by: Cris

Re: New Home Server. Help Needed....... - 18/04/2006 17:48

I have a PD10000 for the firewall/server and haven't noticed a problem like that. I am planning to use an SP13000 for the CCTV machine in good time, but I don't like the sound of that reboot problem, are you running the latest BIOS etc?

I have configured software RAID too, is there any way to see if it's working without removing the server from the rack to disconnect the drive(s)?

Cheers

Cris.
Posted by: andy

Re: New Home Server. Help Needed....... - 18/04/2006 18:04

I'm not running the latest BIOS yet, no. I plan on trying that once my latest RAID rebuild has finished. I also haven't dug around in the BIOS screens yet to see if there is anything there to help.

You could probably disable one of the drives in the BIOS to test a complete drive failure. Failing that you can do a partial test by failing one of the drives in the array using mdadm.

Edit: I've checked, I'm running the latest BIOS
Posted by: andym

Re: New Home Server. Help Needed....... - 18/04/2006 18:37

Quote:
If however I use the power switch on the box to power it off and back on or yank the power it refuses to boot. It powers the main board (the Ethernet link light comes on) but doesn't turn on the VGA output or spin up the drives.


I usually just make sure that resume on AC is selected in the BIOS and usually takes care of it.

Just had my email to say my case has been dispatched.
Posted by: andy

Re: New Home Server. Help Needed....... - 18/04/2006 18:54

Quote:

I usually just make sure that resume on AC is selected in the BIOS and usually takes care of it.



I have that set, it doesn't help. When it is in its "hot" state you hit the power button, the power light comes on but it doesn't event attempt to boot.

Hey, I just cracked 3000 posts
Posted by: andy

Re: New Home Server. Help Needed....... - 29/04/2006 07:06

Quote:

Thanks to your post I finally spurred myself into action and bought the parts for my silent server (to replace the two big noisy and hot servers that currently live in my garage).

I was going to buy my T2e chassis from mini-itx.com, which would have cost me £150. After your post I bought it direct from Tranquil for £65 (via a "best offer" bid) ! Thanks.

I also got a PSU and EPIA SP13000 from them at the same time, with only £10 postage for the lot.



A warning to anyone buying from tranquil_d on ebay, they seem to be hopeless at communication when things don't go quite right. I am trying to get some answers from them with regard to the reboot problems I am having and they also owe me some postage refund. Despite emailing them repeatedly and contacting them via ebay I can't get a response.

I am assuming that tranquil_d is the same as http://www.tranquilpc-shop.co.uk/ so I'll try and phone them next week using the contact number on their website.

I've never left neutral or negative feedback before, don't really want to this time.
Posted by: andym

Re: New Home Server. Help Needed....... - 29/04/2006 08:43

I ordered 3 CF to IDE adaptors and I got an empty jiffy bag from them yesterday. It'd obviously split open in transit. Just sent an email.... let's see how quick they are! Needless to say I'll hold off leaving any feedback for a couple of days.
Posted by: andy

Re: New Home Server. Help Needed....... - 29/04/2006 09:06

I first emailed them 11 days ago now, if emailed again giving my Gmail address in case they were having problems with my normal address.

I wish I could do some troubleshooting to find out which component is at fault (PSU, power brick or motherboard). The obvious thing would be to plug at normal ATX power supply in, unfortunately all my ATX era PCs here are Dells, which means the ATX connector isn't really an ATX connector. I might have to go and buy a real ATX PSU just to test this.
Posted by: Cris

Re: New Home Server. Help Needed....... - 30/04/2006 05:50

Have you tried sending them a message via eBay? That is what I have always done and have always had a speedy responce.

Cheers

Cris.
Posted by: andy

Re: New Home Server. Help Needed....... - 30/04/2006 06:39

Quote:
Have you tried sending them a message via eBay? That is what I have always done and have always had a speedy responce.


Yes, twice.