Home Blog

Let’s connect the home lab to the Internet.

0

Previously on Hemotips

The routers can ping each other and VMs can reach the Internet. I was happy with the results. The happiness did not last for long. The maximum speed the VMs can get is 8 Mbps. The ping latency between the two routers was about 2000 milliseconds. Thus, I wore Sherlock’s hat and pipe and I looked for the suspect. I collected the pieces of evidence and linked all the dots. I was able to find main and only suspect. I pointed my finger to the suspect and I said “Mr. serial link between the routers. You have made the network unstable and killed the Internet connectivity between the poor VMs and the Internet”. He could not say no. I had to replace my 2 2610 routers with a CISCO 1921.

And now back to the topic.

Unlike the routers that are used in the big companies, Home routers do not support the same routing protocols that are used in higher-end routers. As a result, you cannot configure BGP routes between your Cisco routers and home router. The other major problem we are facing is we cannot afford an MPLS nor corporate CPEs. Also, having internet access in your home is crucial if you are practicing services that require Internet Access Like Microsoft WSUS and others.

However, It is possible to connect your home router to your home lab router by doing the following:

  • Make the Internet-facing interface to accept DHCP
  • Create a NAT between your home router and Home Lab router
  • Create access control lists between your internal networks and the peripheral

The network topology will look like something like the following image:

A picture a the topology used to connect the Home Lab to the Internet
The network Topology

Step1: NAT and ACL Introduction

Before heading to the console and getting our hands dirty, let’s talk about NAT first. Nat stands for Network Address Translation and it is used to allow hosts with private addresses to connect to the Internet without providing public addresses for each host in the network. Most of the time, the NAT is overloaded to allow multiple private IP addresses use a single Public IP address. Learn more about NAT overloading in the link here.

ACL stands for Access Control List is one of the methods routers use to control network traffic across different networks within the router. It has the ability to control the traffic down to port level. In other words, it allows blocking SSH as an example from a certain source. ACL has two different types: standard and extended. It is used in our case to allow the internal network to reach the outside of the router. Learn more about ACL in the link here.

Step 2: NAT and ACL Configuration

Before we start, Let’s list the ports and their corresponding IPs:

The portIP Address
GigabitEthernet0/0192.168.100.40/24
GigabitEthernet0/1192.168.10.1/24

In the router IOS running the following commands:

en
conf t
interface GigabitEthernet0/0
ip address dhcp
ip nat outside
exit
interface GigabitEthernet0/1
ip nat inside
exit
ip nat inside source list 1 interface GigabitEthernet0/0 overload
access-list 1 permit 192.168.10.0 0.0.0.255
exit

Horray your home lab network is not connected to the Internet. Please note that this configuration is designed if you have a single network in your home lab. If you are using VLAN or multiple routers in your network, this configuration may not work properly. In a future post, VLAN will be discussed and the configuration will be modified to accommodate these changes. Stay Tuned ūüôā

Bonus: DNS configuration on the router

Even if things are configured correctly, pinging URLs from the router will fail because DNS server is not configured in the router. The following command will configure the DNS server IP router.

ip name-server 8.8.8.8

Home lab upgrade – Routers

0

In the last few days, I upgraded the network setup of my home lab. I have replaced two over a decade old CISCO 2610 routers with a newer router. The previous routers were replaced not for their old age; Until they were decommisioned they were working flawlessly. They were decommissioned due to their limitation. Before the upgrade the setup looks like the following:

 Home Lab Network Infrastructure Before the upgrade
Home Lab Network Infrastructure before the upgrade

As seen in the image above, the connection between the 2610 routers is a serial cable. Serial cable limits the connection speed up to 8 Mbps. Moreover, the connection was not stable at all. As a result, connecting VMs to the Internet is barely possible.

New routers requirements

For a basic infrastructure for my home lab, I do not have niche requirements. Thus, my requirements were dual gigabit ports, not too old and there is room for future ios updates and lastly, it is affordable. I chose CISCO1921-SEC/K9 to replace the current 2610 routers I use. CISCO1921-SEC/K9 is a dual gigabit connection and it is supported by Cisco until 2020. Moreover, it has some security features like VPN connectivity, firewall IOS, and others as well.

Buying the new router

I live in Bahrain and Bahrain does not have a market for used enterprise equipment. As a result, I needed to get my router from outside of Bahrain as I did with my servers. For this purchase, I decided not to spend a lot. Thus, I had a limited budget. I tried buying from US and German-based websites that sell used and refurbished equipment. The shipping cost was too high for the price of the device. Then, I looked into other options like eBay and surprisingly Amazon. I ended up buying my router from Amazon. I bought one router instead of two. The reason is that I do not think about building a two sites architecture for the time being. I paid about $142 (About 43 BD) including shipping!!

Buying a single router will change how devices are connected to each other. The new arrangement will eliminate the need for using dynamic routes (OSPF, EIGRP..) because all networks are handled by a single router. The below image shows how devices are not connected in the home lab.

 Home Lab Network Infrastructure Afterthe upgrade
Home Lab Network Infrastructure After the upgrade

Future Plans

Buying the new router allows me to connect my home lab to the Internet properly (Finally). Moreover, the new setup allows me to deploy services like WSUS and others in my current home lab. Which means more ideas for future blog posts. Also, it allows me to experiment with some of CISCO related security features like VPN and others.

VMDK split files merging: The hard way

0

The previous versions of vSphere supported deploying virtual machines from an OVF template with split VMDK files. However, starting 6.5 VMware removed support for chunked VMDK files. As a result, vSphere 6.5 may not deploy old templates. VMware fixed this issue by releasing a tool that combines VMDK files automatically. Multiple blogs covered this tool like in this post. In this blog post, the same procedure will be covered without VMware tool on Windows and Linux.

Get your Hand Dirty on Linux

1- Combine the chunks

Linux

The process on Linux is simple and it consists of two basic steps. First, combine all VMDK chunks into a single file. To do so, keep all chunks in a single folder and run the following command.

cat vmName-Disk1.vmdk.* > vmName-Disk1.vmdk

Windows

In windows, the step is slightly different. Instead of using cat command like in Linux, copy command with /b argument is used to combine files. Moreover, in the command add all chunks. For example, if disk 1 consists of three chunks. the command will be as following:

copy /b vmName-Disk1.vmdk.000000 + vmName-Disk1.vmdk.000001 + vmName-Disk1.vmdk.000002 > vmName-Disk1.vmdk

2- Edit the OVF file

In your favorite editor (Notepad, Nano, Vim…) open the OVF file. In OVF file you will see something like:

<File ovf:chunkSize="7516192768" ovf:href="vmName-disk1.vmdk" ovf:id="file1" ovf:size=... />

Remove the following part:

ovf:chunkSize="7516192768" 

Then you will get something like:

<File ovf:href="vmName-disk1.vmdk" ovf:id="file1" ovf:size=... />

Finally save the file and you are done. Moreover, remember net to select the metadata checksum file (.mf) is exists as it is no more needed in Vsphere 6.5 and beyond.

Task of the day: Move a file using robocopy

0

A few days ago, I observed something is wrong in the workflow of a unit in a different department. Their workflow requires retrieving a video through FTP and then moving the files to another folder to be transcoded. The process of moving files from a folder to another is currently done manually. This method is not efficient nor proper.

 Problem

There are some issues with their workflow. First, The unit has to perform the process manually, which is a distractive subtask for them. Moreover, the unit does not have any log for the process. Lastly, the files do not have specific arrival times.

The solution: Robocopy

 

 

Screenshot of Robocopy script
Robocopy Syntax

 

A file replication solution solves The unit’s issue. There are many solutions that can be used, but for them I chose Robocopy.

According to Wikipedia, it a command line files and/or folder files replication developed by Microsoft. It is a replacement for Xcopy and it has additional options.  Robocopy allows copying, moving files and folders inside Microsoft Environment. Also, it allows to create a list of files moved or copied and when.

Below, the Robocopy syntax used to fix this issue.

 

 robocopy C:\Source C:\Destination *.* 

 

The upper syntax will just copy all files in the source folder to the destination folder. However, the syntax does not keep a log of all copied files. Robocopy does provide a method to keep a log of all copied files. The below syntax shows how to do so.

 robocopy /log+:"C:\Logs\logs.txt" C:\Source C:\Destination *.* 

/log lets the solution save the operation logs into a certain file. In this example, robocopy saves log file in logs.text appended. It is important to monitor and log every service because it allows system administrations know more about the service when it fails to work properly.

Hemo’s Take

There are many ways to solve this problem. However, I chose this approach because I find easy to understand and implement. Moreover, Robocopy can be used as a part of your backup Strategy.

 

Task of the day: Install CentOS

0

Logo of CentOS
CentOS Logo

Somewhere on earth in an IT department in a company there is a happy and excited junior system administrator at work. The junior is excited and happy because a brand-new server just arrived. Unfortunately, the happiness and excitement didn’t last long after knowing that the server came without OS as requested. What made things worse that our green junior was tasked to install CentOS onthis server with graphical interface.

In this tutorial, you will cover the process of installing CentOS, so the friend of our friend will not have a mini panic attack at work.

 

Pre-installation notes:

There are some things should be take into considerations before doing the installation process. First, make sure the hardware meets the minimum requirements of CentOS. Second, make sure the server is configured that it reads from the boot disc or PXE server. Lastly, you need a mouse and a keyboard to proceed in the installation.

Let’s install CentOS:

From main¬†menu choose “Install CentOS 7” and wait for the next image

 

Select the language of the operating system. Remember, this is not the keyboard layout. Press Next to proceed.

In this windows you will meet Installation Summary where you will configure Time and Date, Installation Destination, Network and Host Name and Software Selection. You will start with Date and Time, thus click on it.

Next, select your city and timezone correctly. Do you not worry about network as you will fix it in a later stage. Press Done to go back to Installation Summary window.

In Installation Summary window click Installation Destination.

Our senior told us that he needs a single partition in a single drive. As a result, there is nothing to be done here. So,  press Done.

Next,  configure Network and Host Name, so click on it.

In this window there are two main task to be done. First, you need to enable the network port by clicking On to turn it on and get an IP address from a DHCP server. Second, you need to give our server a proper host name. I named it “centos.hemotech.tips”. Finally, click on Done to go back to Installation Summary window.

Before proceeding to the next step,  go back to Date and Time and  enable Network Time and then  click Done.

Lastly,  configure the most critical step in the Installation Summary window. The step is Software Selection, so in the Installation Summary click on it.

By default, CentOS come in the Minimal Install option. Minimal Install option means that it will be installed without GUI and the minimal application. To install GNOME, click on GNOME Desktop and selection the same option as in the below image. Then click Done.

In the Installation Summary click Begin Installation.

In this step, please insert the root access password. You shouldn’t add a weak password in a production server, but if you choose a weak password as I did, you need to press Done twice.

Next, you will create another administrative account. You should not use root account at any time.

Post Installation step. We are almost there ūüôā

Reboot the system when prompted.

After the reboot the you will this list. Make sure you pick the one that does not have the work rescue.

In the License Information window, check the check box and click Done.

Finally, click on Finish Installation.

You do not need to change the settings here, so press Done or Skip.

You did it ūüôā

Hemo’s Take:

Installing an operating requires a lot of steps, yet these steps are easy to understand and follow. Moreover, it critical for any system administrator to know how to install a Linux operating system.

 

Hemo and buying a used server

0

A humor Image of a monkey looking at a computer screen.
Just a Random Image XD

Months ago I decided to finally build my personal home lab. I decided to build mine so I can develop my skills and become more familiar with breaking servers. As a result, I allocated some budget for the project that does not exceed 1000 BHD ( about $2600). My requirements are a server enclosure ( Server Rack) that is no more than 30U, a single U used server and few switches and routers for the first phase.

In this blog post I will cover my experience of getting a used server in Bahrain. In other countries the experience will be totally different.

Getting the rack

I tried to get a server rack as cheap as possible. I was looking for a used worn out rack for month but no luck. After that, I scouted most IT shops in the country for an affordable one. However, the cheapest one I found was for about 150 BHD ($380). A godsend friend who works for an IT vendor in the region used his networking to allow me to have a new rack for close to free.

The used server

Buying Locally

After securing a rack to store the equipment I started looking for used servers. I chose getting a¬† used one because I don’t need brand new server in the server lab. After some time I found a used server for about 1000 BHD, which is way above my budget. After giving up finding a used to for an affodable price in the kingdom, I looked for low end new server and I found one for an affordable price (Sort of).

Buying Online

Logo of servermonkey used server seller
Server Monkey logo

 

Before going to buy the weak server, I googled for companies that sells refurbished servers. I found a chunk of them, but I decided to go for ServerMonkey. ServerMonkey sells refurbished hardware that is pulled from working environment then they refurbish them to  give those hardware a second life. To my surprise server cost and shipping from The States to Bahrain is less than buying a refurbished one and the imported one is better and newer than the local one.

When I got my server the packaging was really good and saved the equipment for crashing with each other all the way from US to here. Installing the server in the rack went with no issues thanks to some videos on YouTube. Finally I connected my Server into my PDU.

Hemo’s Take

Unfortunately, there is not a single community that manages refurbishing used equipment in the country. Also retailer here should stop selling used equipment at a ridiculous prices. People who are buying these stuff are not business, but people building their home server labs. Be mercy on them.

Introduction to Linux

0

Picture of Linux mascot
Linux mascot “Tux”

As a tech worker, one of the skills¬†that is a must is ability to use Linux. I won’t give you a¬†walkthrough on how to write your first command or how to install it. However, I will talk about what do you need to know about Linux before using it. It is because Linux is not just an¬†operating system. It is¬†a¬†philosophy and an interesting story in a form of an operating system.¬†

What is Linux? 

Linux is an open source operating system created by Linus Torvalds for x86 architecture and then for other architectures. Linux is greatly seen and used in data centres due to its performance, stability and flexibility. 

Open Sauce ?!! 

No not open sauce but open source. In software development world there are two different philosophies. The first believes no one should ever see how a certain application is made and nothing of its source code is open to public. This philosophy is called closed source or proprietary code.  The other school believes that the source code must be open for public and they have the modify the code for their need, as long as they give credit to the original author. 

Linux History

In 1991 when Linus¬†Torvalds¬†was still as student he found out that there isn’t a free and open sourced operating system that available for the public.¬†So, he decided to build a new operating system that¬†free and opensource.¬†

Linux in depth… slightly

Over the years Linus is pretty much different than what it was in 1991. Also, due to availability of its code for the masses, there are many flavors of Linux. Each flavor is called a distribution. Each distribution differs from other by having different collection of application that are added to the system.  

Distributions are split into two families; RedHat and Debian. Each family has its own characteristics. RedHat is a commercial Distribution that is free but you have to pay for it for its support. RedHat is also known for its stability and production ready. However, its update cycle is usually slower than the Debian family. Debian family is entirely free and tend to update every 6 months. Moreover, it is equipped with the latest technologies and applications. 

Hemo’s Take

There isn’t the best distribution of Linux, because each distro has its benefits. As a result, you should choose the distro that matches your needs. Thus, I advise you to play with various distros until best fit is found and chosen.

Consumer Hardware VS Server Hardware

0

One of the mistakes some novice IT pros do is suggesting using high-end consumer hardware instead of Server grade hardware. They suggest using high-end consumer grade hardware because it has better performance with the price range. However, price tag is just a factor in calculating overall cost of a certain asset. In this post, we will see why server-grade hardware is more beneficial than high-end consumer hardware.

 

Designed to last longer and more reliable!!

Server hardware is designed to have a longer time span that consumer hardware, because servers in general stay in production for a longer time than consumer hardware. Moreover, servers are designed to be on 24/7, unlike personal machines that most people turn them off for hours daily.

Mean time between failure or MTBF is a unit that measure the average time a component will take to failure under optimal operation conditions. As an example, according to western digital WD Gold MTBF is 2.5 million hours.  On the other hand, consumer hard drives do not come with MTBF, however they tend to fail after 500 thousand hours.

Any enterprise will have more profit if they used their equipment for a longer time and less down time, because their ROI will be higher.

Fault tolerant

The other reason IT pro choose servers grade hardware over consumer grade hardware is its high fault tolerant and redundancy. As an example, servers come with two PSUs, to prevent downtime because of a dead PSU.

Fault tolerance allows devices to run even if one or more of its components is faulty. As a result, no service will be interrupted and sysadmins will be happy.

Support

Support provided to server grade hardware is often better than consumer grade hardware. Server grade hardware comes with better support packages, such as 24/7 support and on-site equipment replaced done by support representative. Moreover, vendor dedicate a certain specialist to large customers to ensure the representative is familiar with production environment and provide faster and better service.

 

Hemo’s Take

‚ÄúIt depends‚ÄĚ is what is answer should be. As a system administrator, you should evaluate the value of each service and decide does it worth the support it comes with or not. Also think about the time and resource wasted from the down time.

Server 101: SAN and NAS

0

Storage is one of the topics that is done differently in the corporate world than home environment. In SOHO data is usually stored in directly attached hard drives and in a device in a device over the network using a NAS. On the other side, corporate usually depend on something called a SAN to store their corporate data.

NAS?

Image of a NAS
Synology DS1511+ NAS – Courtesy of Synology

 

NAS stands for Network Attached Storage, which is basically generally a device over the network that is used to host and share information over the network. The device can serve multiple services or dedicated for file storage.

NAS is a file level storage. File level storage mean that the NAS and the client exchange files between them over the network, just like when a file download from a website. Moreover, with file level storage files are usually stored in a fixed location with a fixed storage capacity. As a result, expanding storage capacity is usually not possible and always require some downtime.

There are many protocols used for files storage such as SMB and AFP. Most NAS devices nowadays support and use SMB as its main file exchange protocol.

SAN?

 

A picture of a SAN Component
emc VNXe1600 SAN – Courtesy of EMC

SAN stands for Storage Area Network and it is a network of multiple devices connected to serve a single purpose …storage. The network contains fiber matrix, storage controller and even pools of blazing fast hard drives.

Unlike NAS, SAN uses block level storage. In a nutshell, block level storage means that communication between the SAN and the client is treated like a directly attached hard drive into a motherboard. As a result, SAN mounts are visible as a local hard drive on clients.

There are two main protocols for SAN: iSCSI (Internet Small Computer Systems Interface) and FC (Fibre Channel). Most SAN nowadays use FC is built specifically for SAN and it is faster than iSCSI, but it is more expensive.

Why SAN over NAS?

Large enterprises use SAN over NAS for multiple reasons. First, SAN performance is higher than most NAS devices, due to the way storage area networks are built. Second, SAN allow easily to modify storage allocation of each storage mount. Also, the mount can reside on single hard drive, single virtual hard drive and event on two different building! Third, Storage are networks are real fault tolerant if built correctly. HP has proofed that a SAN can be bullet proof literally in an old TV commercial!!! The commercial can be viewed bellow.

 

Hemo’s Take

It is alright if you are not very familiar with SAN, as it is considered as big boys toys. Myself, I don’t have the permission to work with it yet, but one day both of us will work with it ?

Automating Operating System Installation

0

Screenshot of MDT
Courtesy of Microsoft

Operating system installation is one of the most basic and simple process done by sysadmins or help-desk people, yet it is one of the most expensive processes for any business, because it wastes IT staff time and employees time. Besides being expensive and time consuming, it is also very repetitive and prone to errors.

The problem

Let’s image the following, your business decided to build an onsite training center at work with capacity of 25 trainees. The new center needs a new machine for each trainee in the center. Moreover, these machines come without an operating system, because you already have an enterprise license from Microsoft. Accomplishing this task will keep the entire team busy for the entire day!  By the end of the day, you will have a dead tired team who spent an entire day on a single task, and training team who were idling due to uninstalled machines.

The traditional way the team is doing this task is by grabbing an installation disc and fill the prompted instruction, until the end. During this process, the staff member must be next to the machine to answer the required questions. The actual process can take up to an hour per machine.

The biggest problem with operating system installation is not the time it consumes, but the IT team might have 25 machines with 25 different configurations. Such difference in configuration may lead into future problems in the future.

Such process can be enhanced by complying two main principles. First, Automating the processes as much as possible. Second, unifying all the hardware and software as much as possible. Operating system installation automation complies these principles.

Automation?

In a nutshell, operating system installation is done using through running a PXE server, and creating a proper answer machine file which is used answer question asked during installation.

Available solutions

They are different solutions used for operating system automation. Microsoft suggests Microsoft Deployment Toolkit to automate their operating systems. Red Hat released Kickstart to automate its operating systems.

Hemo’s Take

Cloning is also a solution, but I suggest not to use it. The master image of the clones needs to be up to date to have all the chnages. Also, SA needs to run sysprep to change GUID and SID of each clone. Finally, cloning hides the history of the process. As a result, the SA does not know installation history.