Category Archives: Amazon Web Services

XWindows on EC2

Launch Xming server on local desktop (Windows)
Turn on X11 forwarding in putty.
(I did not have to set any local DISPLAY environment variables)

sudo yum install xorg-x11-xauth.x86_64 xorg-x11-server-utils.x86_64 xterm

I think I logged out and then ssh’d back in at this point (with X11 forwarding on).

Then I just did an “xterm”. That should open a new terminal window. From that you should be able to run commands that need to open new windows without having to set additional environment variables. The window took a bit to open. If it doesn’t pop up, look at your task bar to make sure it didn’t start minimized. I think that may be an Xming thing…

For a really slow browsing experience (or to run an installer that needs a web browser):

sudo yum install firefox

Then run firefox from the xterm….

AWS Linux EC2 Instance Creation Checklist

This is intended to be an ongoing check list of tasks (with implementation notes) to perform when creating a new Linux instances on AWS

  1. Choose OS (I am trying to use Ubuntu for instances that don’t specifically require RHEL )
  2. Create the instance with desired root volume size.  RHEL seems to create a volume with the specified size, but then allocates a 6GB root partition.
  3. Choose appropriate VPC, subnet (considerations: public, private, AZ, what RDS or other instances need to communicate with it)
  4. Select the appropriate Key (will create additional accounts and assign keys as needed)
  5. Select or Configure Initial security rules
  6. Start instance
  7. Resize root partition if necessary (currently only necessary for RHEL 7 instances)
    1. Stop the instance
    2. If you have done any configuration already, take a snapshot of the volume to protect your work (only if restoring the snapshot would be worthwhile should something go wrong)
    3. Note the mount point and volume name of the root volume
    4. detach the volume and attach it to another instance (preferable running the same OS – spin a temporary one up if needed).  You do not need (nor want) to mount it.
    5. See: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage_expand_partition.html#prepare-linux-root-part  (read this to verify the next steps still hold – and to see example output)
    6. make sure that the filesystem is OK BEFORE you resize it.  (the URL above skips this step):   xfs_repair /dev/xvdf1  (for xfs used by RHEL 7) OR for ext4: sudo e2fsck -f /dev/xvdf1  (or whatever device you attached the volume as)
    7. parted /dev/xvdf (or whatever device you attached it as)
    8. (parted) unit s
    9. (parted) print
    10. (parted) rm 1
    11. (parted) mkpart primary 2048s 100% OR (for gpt partitions) (parted) mkpart Linux 4096s 100%
    12. (parted) print
    13. (parted) set 1 boot on
    14. (parted) quit
    15. xfs_repair /dev/xvdf1  (for xfs used by RHEL 7OR for ext4: e2fsck -f /dev/xvdf1  (or whatever device you attached the volume as)
    16. detach the volume and reattach it in the proper place on the new instance
    17. start the instance
  8. Allocate desired swap space as a swap file
    1. This example will assume an 8 GB swap file (/var/swap.1)
    2. sudo  /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=8000
    3. chmod 600 /var/swap.1
    4. sudo /sbin/mkswap /var/swap.1
    5. sudo /sbin/swapon /var/swap.1
    6. To enable it by default after reboot, add this line to /etc/fstab: /var/swap.1 swap swap defaults 0 0
  9. Configure hostname
    1. hostnamectl set-hostname <hostname.FQDN>
    2. add hostname and FQHN to /etc/hosts
    3. modify /etc/cloud/cloud.cfg and comment out the set_hostname, update-hostname, and update_etc_hosts lines
  10. Set timezone
    1. (RHEL 7)  timedatectl set-timezone America/Chicago
    2. ln -sf /usr/share/zoneinfo/America/Chicago /etc/localtime    (verify that timedatectl did this for RHEL 7 – if not, do it!)
  11. Create additional accounts
    1. groups ubuntu (or ec2-user)
    2. useradd -s /bin/bash -m -d /home/newuser -G adm,cdrom,floppy,sudo,audio,dip,video,plugdev,netdev newuser
    3. edit /etc/sudoers:
      newuser ALL=(ALL) NOPASSWD:ALLOR change%sudo ALL=(ALL:ALL) NOPASSWD:ALL  (put NOPASSWD: into existing line – for some reason this works for ubuntu but not for the second user without spec’ing this)
    4. mkdir ~newuser/.ssh
    5. vi ~newuser/.ssh/authorized_keys
    6. add in the contents of the newuser.pub file from the my repository
    7. chmod 700 ~newuser/.ssh
    8. chmod 600 ~newuser/.ssh/authorized_keys
    9. chown -R newuser:newuser ~newuser/.ssh
    10. test logging into the account from another machine and verify that “sudo bash” works.
  12. Allow passwords for SSH (ONLY if required/desired)
    1. edit /etc/ssh/sshd_config and set PasswordAuthentication yes
    2. restart sshd
  13. Create and mount additional volumes
  14. Configure automatic backups
    1. set the Autosnapshot tag to True for all volumes that you want backed up automatically
  15. Install additional software
  16. Do initial software update
    1. apt-get update && apt-get dist-upgrade
    2. yum update
  17. Configure automatic updates
    1. RHEL: yum install yum-cron
      1. for RHEL < 7, edit /etc/sysconfig/yum-cron
      2. /etc/init.d/yum-cron start
      3. chkconfig yum-cron on
      4. For RHEL >= 7, edit /etc/yum/yum-cron.conf
      5. if you only want security updates => update_cmd = security
      6. apply_updates = yes (also download_updates)
    2. Ubuntu:
      1. apt-get install unattended-upgrades
      2. dpkg-reconfigure -plow unattended-upgrades
      3. verify /etc/apt/apt.conf.d/20auto-upgrades
      4. verify /etc/apt/apt.conf.d/50unattended-upgrades (verify reboot time and that automatic reboot is true)
  18. Tag volumes for automatic backup
    1. set the Autosnap tag to True
  19. Configure and test additional security rules
  20. Create a restore AMI!  (Since this creates a snapshot, I don’t <think> this will take up a lot of extra space for instances that we plan to backup anyway.)  This is a REALLY good idea since restoring a snapshot relies on you to first create a new instance based off the same AMI as the original and then replace or attach the volume on that instance.  We can’t guarentee that the original instance will still be available.

OpenVPN on AWS

So far I am impressed with OpenVPN AS on AWS with Windows clients.  I haven’t tested beyond the two test users — so how well it scales is yet to be seen.

I am having some trouble getting a Linux host (Ubuntu 14.04) to function properly.  I’m not seeing errors, but the connection doesn’t appear to be working.  That appears to be high on my “ToDo” list for today.

OpenVPN Notes

Trying OpenVPN on Ubuntu 14.04 EC2 instnace. Warning these are mostly just notes to myself — use with caution. That includes you, future me…

TLDR version: If it absolutely comes down it it, I think I could eventually make the community version do what I need. However, with more than just this on my plate, it is probably more cost effective (and sane) to go with the commercially supported one. Too many irons in the fire…

References:
https://help.ubuntu.com/14.04/serverguide/openvpn.html
http://www.linuxfunda.com/2013/09/14/how-to-install-and-configure-an-open-vpn-with-nat-server-inside-aws-vpc/

I will note that I did have my hostname set to the desired value going into this. It might make a difference since I’m going to be making keys…

sudo bash # because I’m lazy….
apt-get update && apt-get dist-upgrade
apt-get install openvpn
apt-get install easy-rsa

root@vpn:~# whereis openvpn
openvpn: /usr/sbin/openvpn /etc/openvpn /usr/lib/openvpn /usr/include/openvpn /usr/share/openvpn /usr/share/man/man8/openvpn.8.gz

mkdir /etc/openvpn/easy-rsa/
cp -r /usr/share/easy-rsa/* /etc/openvpn/easy-rsa/

vi /etc/openvpn/easy-rsa/vars  (straight from the Ubunut help article referenced above — I modified the first 5 lines and left the last three as is.)

export KEY_COUNTRY="US"
export KEY_PROVINCE="NC"
export KEY_CITY="Winston-Salem"
export KEY_ORG="Example Company"
export KEY_EMAIL="steve@example.com"
export KEY_CN=MyVPN
export KEY_NAME=MyVPN
export KEY_OU=MyVPN

root@vpn:~# cd /etc/openvpn/easy-rsa/
root@vpn:/etc/openvpn/easy-rsa# source vars
NOTE: If you run ./clean-all, I will be doing a rm -rf on /etc/openvpn/easy-rsa/keys
root@vpn:/etc/openvpn/easy-rsa# ./clean-all
root@vpn:/etc/openvpn/easy-rsa# ./build-ca
error on line 198 of /etc/openvpn/easy-rsa/openssl-1.0.0.cnf
140316838659744:error:0E065068:configuration file routines:STR_COPY:variable has no value:conf_def.c:618:line 198

DOH!
line 198 of /etc/openvpn/easy-rsa/openssl-1.0.0.cnf:
# This stuff is for subjectAltName and issuerAltname.
# Import the email address.
# subjectAltName=email:copy
subjectAltName=$ENV::KEY_ALTNAMES

vi /etc/openvpn/easy-rsa/vars

# Added to fix error on build-ca line 198 of openssl-1.0.0.cnf seems to want
# a environment variable KEY_ALTNAMES — not sure what it does
# gonna give it something memorable in case it comes up later…
export KEY_ALTNAMES=”BoogaBooga”

source vars
./clean-all
./build-ca

./build-key-server

I accepted the defaults as I had already customized the text config file…

I did my initial attempt with the default values (except the last two that require you to hit y). This means that I did NOT include
a challenge password. That is something I may want to look into..

./build-dh (Note that this use 2048 bits by default rather than 1024)
cd keys/
cp .crt .key ca.crt dh2048.pem /etc/openvpn/

Build each client key (these should ultimately be removed from the server and exist only on the client machines…):
I think I will need to do this each time a new client machine is add (ick — need to script this out…)

cd ../ (cd /etc/openvpn/easy-rsa/)
source vars
./build-key client1

Server Config (From included sample files):

cp /usr/share/doc/openvpn/examples/sample-config-files/server.conf.gz /etc/openvpn/
cd /etc/openvpn
gzip -d server.conf.gz

vi server.conf:

ca ca.crt
cert .crt
key .key # This file should be kept secret

# Substitute 2048 for 1024 if you are using
# 2048 bit keys.
dh dh2048.pem

service openvpn restart (Just to make sure an old version wasn’t running)

root@myvpn:/etc/openvpn# ifconfig tun0
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.8.0.1 P-t-P:10.8.0.2 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

root@myvpn:/etc/openvpn# lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
extra services
openvpn 2437 root 6u IPv4 11132 0t0 UDP *:openvpn

tar cvzf ~/client1.tgz client1.crt client1.key ca.crt

then scp those to the client for testing

on the client:

sudo apt-get install openvpn
sudo cp /usr/share/doc/openvpn/examples/sample-config-files/client.conf /etc/openvpn/

sudo cp ca.crt client1.crt client1.key /etc/openvpn

vi /etc/openvpn/client.conf

remote my.vpn.svr.ip 1194
ca ca.crt
cert client1.crt
key client1.key

root@fizban:/etc/openvpn# service openvpn restart
* Stopping virtual private network daemon(s)… * No VPN is running.
* Starting virtual private network daemon(s)… * Autostarting VPN ‘client’

root@fizban:/etc/openvpn# ifconfig tun0
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.8.0.6 P-t-P:10.8.0.5 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

root@fizban:/etc/openvpn# ping 10.8.0.1
PING 10.8.0.1 (10.8.0.1) 56(84) bytes of data.
64 bytes from 10.8.0.1: icmp_seq=1 ttl=64 time=106 ms
^C
— 10.8.0.1 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 106.110/106.110/106.110/0.000 ms

So, now I have a vpn server that can talk over a tunnel to 1 client. At this point the client can ONLY talk to the VPN server….

So now I have to figure out IPTables…
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

if all goes well, I will :
service iptables save (that’s actually RHEL I think)
Ubuntu:
apt-get install iptables-persistent
service iptables-persistent start

vi /etc/sysctl.conf
-> uncomment: net.ipv4.ip_forward=1

sysctl -p

copy the configure-pat.sh from my nat instance (so I think it originates from the Amazon NAT AMI):

#!/bin/bash
# Configure the instance to run as a Port Address Translator (PAT) to provide
# Internet connectivity to private instances.

function log { logger -t “vpc” — $1; }

function die {
[ -n “$1” ] && log “$1”
log “Configuration of PAT failed!”
exit 1
}

# Sanitize PATH
PATH=”/usr/sbin:/sbin:/usr/bin:/bin”

log “Determining the MAC address on eth0…”
ETH0_MAC=$(cat /sys/class/net/eth0/address) ||
die “Unable to determine MAC address on eth0.”
log “Found MAC ${ETH0_MAC} for eth0.”

VPC_CIDR_URI=”http://169.254.169.254/latest/meta-data/network/interfaces/macs/${ETH0_MAC}/vp
c-ipv4-cidr-block”
log “Metadata location for vpc ipv4 range: ${VPC_CIDR_URI}”

VPC_CIDR_RANGE=$(curl –retry 3 –silent –fail ${VPC_CIDR_URI})
if [ $? -ne 0 ]; then
log “Unable to retrive VPC CIDR range from meta-data, using 0.0.0.0/0 instead. PAT may ma
squerade traffic for Internet hosts!”
VPC_CIDR_RANGE=”0.0.0.0/0″
else
log “Retrieved VPC CIDR range ${VPC_CIDR_RANGE} from meta-data.”
fi

log “Enabling PAT…”
sysctl -q -w net.ipv4.ip_forward=1 net.ipv4.conf.eth0.send_redirects=0 && (
iptables -t nat -C POSTROUTING -o eth0 -s ${VPC_CIDR_RANGE} -j MASQUERADE 2> /dev/null ||
iptables -t nat -A POSTROUTING -o eth0 -s ${VPC_CIDR_RANGE} -j MASQUERADE ) ||
die

sysctl net.ipv4.ip_forward net.ipv4.conf.eth0.send_redirects | log
iptables -n -t nat -L POSTROUTING | log

log “Configuration of PAT complete.”
exit 0

I need to run this from something akin to rc3.d/S99local (add that to my todo list)

I copied this file to /usr/local/sbin/configure-pat.sh and added it to /etc/rc.local (which seems like the place to put it for Ubunut 14.04).

Then I’m gonna reboot and see if anything works. I may have to configure routes for my subnets in aws???

You know it is at this point, where the configuation ceases to be fun. I started looking more at the commercial version. Given that I have killed an entire afternoon and still have only the server and one client talking to each other, the price of the licenses is starting to look good!

Long term, I’d like to come back and work on this some more, but for now, it just isn’t cost effective to spend more time on it…

AWS and Windows AD networks

I am starting on my first “real” design of a Windows AD forest on AWS.  I’m attempting to do it using some of the recommended approaches.

I’m using a mostly private network with NAT and RDP gateways across multiple availability zones.  I’m also trying to plan for future network growth and additional functionality as the use case grows.  Far too often, tight schedules end up causing designs to “happen” rather than be researched and planned.

Some days I feel like I’m trying to walk a rather large, determined, and eager Golden Retriever — while wearing roller-skates…

FYI: Me + roller-anything = REALLY bad idea.