Tips for installing memcached on Red-Hat 4

1-) web access :)

2-) rpm -ihv ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel4/archive/x86_64/libevent-1.3e-4.el4.pp.i386.rpm

3-) rpm -ihv htp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel4/archive/i386/memcached-1.4.5-1.el4.pp.i386.rpm

Dont install memcached 1.2 versio0n and the stupid libevent 1.3b beta release than you will get EAGAIN resource temporyliy unavailable erro when trying to asyncroushly write to memcached server.

Our Coreopsis Rota Video Monitoring Deprecated Latest release

We stopped developing the Coreopsis Project on 2007. This is the latest release of our Windows Based Ip Camera management system (2007). From the below link download Setup1_05.zip. Only Windows XP Windows 2000 and Windows 2003 servers are supported. The software is for free....

http://code.google.com/p/barisergunexperimentalstuff/downloads/list?can=1&q=

Memcached Performance Tests and Source Code (Java Source, Ant Project)

What is Memcached?

I have prepared some test cases in order to test the performance of Memcached distributed caching library. You can get them from my code repository with the following mercurial script.

hg clone https://barisergunexperimentalstuff.googlecode.com/hg/ barisergunexperimentalstuff

For your information if you don't have a mercurial rep client than you can get it with your favorite package manager. I use "sudo apt-get install tortoisehg" on my Ubuntu.

A-) Local tests:

Local tests are done with a sample DataObject. when you run the tests 8 different memcached instances are started on 8 consequent ports.

Steps to follow

- Install memcached (eg. sudo apt-get install memcached). On my previous post I also have information about on installing memcached on Centos 5.5

- Then, run tests with below ant script

ant local.test -Dentry.count=10000 -Dttl=60

Dentry.count is used to configure number of objects to cache. Dttl is used to configure TTL for each object in the cache.

A-) Distributed tests:

Local tests automatically starts and terminates the memcached servers on local machine. For distributed tests you will have to start the memcached servers on remote machines and give the server_ip:server_port combinations with a specific format as shown below.

Steps to follow:

- Install memcached on remote servers

- Start memcached servers.

Eg:

memcached -d -u root -m 1024 10.34.3.13 -p 1121

- Run distributed tests with following arguments(just a sample)

ant distributed.test -Dentry.count=10000 -Dget.timeout=10 -Dttl=300 -Dservers="10.34.3.11:1121 10.34.3.12:1121 10.34.3.13:1121"

Dget.timeout is for timeout for reading from cache
Dservers is list of memcached servers.

Tips for installing memcached on Centos 5.5

1-) Get the below memcached rpm file

wget ftp://ftp.pbone.net/mirror/ftp.pramberger.at/systems/linux/contrib/rhel5/i386/memcached-1.4.5-2.el5.pp.i386.rpm

My architecture was i386 you find yours an get the appropriate one.


2-) rpm -ihv memcached-1.4.5-2.el5.pp.i386.rpm

There you go....

Performance of inserting data on Jboss Cache compared to ConcurrentHashMap.

You can use the below Beanshell script directly with the Jboss Cache 3.2.5 GUI Demo with embedded Beanshell. Just copy the below code piece to a file eg trial.bsh. And to run it from the embedded Beanshell of JBoss Cache GUI Demo do -> source("trial.bsh"). You should see some tps and performance related text output. The below beanshell script file is for ENTRY_COUNT = 120000; You can play with different number of entries. Also do not forget to play with Xms arguments not to get a stack overflow for large entries.


import org.jboss.cache.Fqn;
import java.util.Date;
import java.io.*;
import java.util.concurrent.ConcurrentHashMap;
import java.lang.instrument.*;



public class LocationDataObject implements Externalizable {

private static final long serialVersionUID = 3383966388917008053L;

private Double xCoord;
private Double yCoord;
private Date timeStamp;

public Double getXCoord(){
return xCoord;

}
public Double getYCoord(){
return yCoord;
}
public Date getTimeStamp(){
return timeStamp;
}

public void setXCoord(Double xCoord){
this.xCoord =xCoord ;

}
public void setYCoord(Double yCoord){
this.yCoord = yCoord;
}
public void setTimeStamp(Date timeStamp){
this.timeStamp = timeStamp;
}


public void readExternal(ObjectInput in)
throws IOException,ClassNotFoundException {

xCoord = in.readDouble();
yCoord = in.readDouble();
timeStamp = (Date)in.readObject();
}


public void writeExternal(ObjectOutput out)
throws IOException {
out.writeDouble(xCoord);
out.writeDouble(yCoord);
out.writeObject(timeStamp);
}

};



public class TestJbossCachevsHasmap{
private static final Long ENTRY_COUNT = 120000;

private static Instrumentation instrumentation;

public static void premain(String args, Instrumentation inst) {
instrumentation = inst;
}


public static void main(String [] args){

childFqn1 = Fqn.fromString("/Msisdn");

tm = cache.getTransactionManager();

tm.begin();
msisdnMap = root.addChild(childFqn1);
tm.commit();

locData = new LocationDataObject();

Long startTime = System.currentTimeMillis ();

for(Long fakeMsIsdn=0; fakeMsIsdn < ENTRY_COUNT; fakeMsIsdn++){

locData.setXCoord(10.00D);
locData.setYCoord(20.00D);
Date dateNow = new Date();
locData.setTimeStamp(dateNow);

tm.begin();
msisdnMap.put(fakeMsIsdn.toString(), locData);
tm.commit();

}

Long endTime = System.currentTimeMillis ();

Long executionTime = (endTime - startTime);

double tps = (ENTRY_COUNT*1000)/executionTime;

System.out.println("Duration of Jboss Cache for inserting " + ENTRY_COUNT + " entries in " + executionTime + " msec. Average tps is " +tps );

//System.out.println(ObjectSizeFetcher.getObjectSize(root));

concurrentMap = new ConcurrentHashMap ();

Long startTime2 = System.currentTimeMillis ();

for(Long fakeMsIsdn=0; fakeMsIsdn < ENTRY_COUNT; fakeMsIsdn++){


locData.setXCoord(10.00D);
locData.setYCoord(20.00D);
Date dateNow = new Date();
locData.setTimeStamp(dateNow);
concurrentMap.put(fakeMsIsdn.toString(), locData);

}

Long endTime2 = System.currentTimeMillis ();

Long executionTime2 = endTime2 - startTime2;

double tps2 = (ENTRY_COUNT*1000)/executionTime2;

System.out.println("Duration of ConcurrentHashMap for inserting " + ENTRY_COUNT + " entries in " + executionTime2 + " msec. Average tps is " +tps2 );


}

public static long getObjectSize(Object o) {
return instrumentation.getObjectSize(o);
}



}


TestJbossCachevsHasmap.main(null);

Integrated Mockito TestNG and Emma on an ant built (Targeted for Java developers who aim TDD)

I have prepared a very simple sample ant project showing how to integrate and run Mockito, TestNG and Emma projects. For those of you who wonder what these projects are, Mockito is Java Mock framework, TestNG is Unit Test Case Framework and Emma is the code test coverage tool.

The purpose of this project is to demonstrate how ClassThatOutputsHelloMessage is tested with TestNG. How NameProviderInterface is mocked with Mockito and how is the Mockito syntax written in TestNG test case. Showing dependency injection = Giving NameProviderInterface as ClassThatOutputsHelloMessage constructor argument. And what kind of code coverage report you obtain with emma.

Inorder to see the sample checkout my experimental stuff rep with hg(mercurial client)

hg clone https://barisergunexperimentalstuff.googlecode.com/hg/ barisergunexperimentalstuff

And than, navigate to testngmockitosimplesample folder. And run the below ant command:

ant runtestswithemma

If everything is okey you will see a generated folder named coverage. Inside this directory you will see 3 different formats of report(html,xml,txt). Especially check the html format to easily navigate the report to see which lines of code is not tested.

At the begining you will see that not all of the code is tested. In the ClassThatOutputsHelloMessageTest.java source Uncomment the commented-out test case. And re-run tests with emma. Check and verify that the coverage report has changed with %100 coverage.

Please place a comment here on this blog if you face any problems.

Things to remember:

Make sure your environment variable JAVA_HOME is set. When you try to integrate these tools to your own project keep in mind that you should compile your sources with debug="on" option in order to be able to see the line coverage of your codes with emma instrumentation.


Blogs that helped me:
http://rwatsh.blogspot.com/2008/03/using-testng-with-emma-for-automating.html

Imagine how Test Driven Development helps you.

I decided to write this article to be able to provide a general understanding about test driven development and how it may help you. When I am talking about the subject I will refer to bugs as real world bugs like a single fly or a mosquito etc. and Testing approaches as weapons used to kill them ie. smashing, electrical killers or an axe (well can you kill a mosquito with an axe, theoretically yes you can; yet we will see the results ;)) I will number these testing approaches from 1 to 3 and finally return back to the real world mosquito scenario.

1-) Unit Testing with Mock APIs: (SMASH)

Lets start with unit testing and mock APIs around them. When we need to talk about unit testing keep in mind there other concepts such as Fake APIs as well. Well to give you an understanding about the difference of these 2 approaches:

a-) For Mock API you can use mock frameworks such as (Mockito for Java Google Mock for C++) and all you will have to do is write the behavior of your mocked API during your Unit Test cases as pre-compiled expectations and you will spend no or minimum (minimum for Adapting API implementations) effort on implementing the mocked api. Because mock framework does the implementation for you, or better to say; mock framework provides you an API as if there is a real API interacting with your unit or module under development.

b-) Fake API approach is much more time consuming approach for a developer. Because based on your real API you have to implement a "Fake API" which behaves like your real API and you will have to provide switch like structures to manipulate the behavior of the implemented fake api.

Well considering from development point of view; mock approach seems much more reasonable compared to the fake approach. But you have to be careful about the design of your software module if you follow the mock approach. The mock approach requires you to provide a visibility for 3rd party APIs from out of your implemented module. Here the Dependency Injection is the remedy for such design considerations which I will discuss about on another article.


Pros:
- Isolation of development modules completely from each other during design.
- Unit tests for a module run in seconds or mseconds.
- You can find simple bugs early in the development cycle.
- Maintenance and refactoring efforts of the software module is significantly reduced.
- The developer is more confident about his software modules.
- You do not need a test bed.
- The unit tests can be %100 percent automated.

Cons:
- Development effort of software developer may have been doubled during the initial development. (considering reduction of maintenance efforts in later cycles this negative affect is balanced in whole life cycle of the software module).
- Another designing requirement of the software module rises such as "Design From Test Point Of View".


2-) Integration Testing:(ELECTRICAL BURNER FOR KILLING INSECTS)

Now that each software module has been developed in an isolated kind of approach, it is time to bring them together or all together and see the affects of them on each other. There are some commercially available tools beside home made integration test frameworks which software developers can use for such purposes. You can think of this stage as replacing the mock apis (during unit tests) with real implemented modules by other developers. Generally an integrator such as the technical leader of the project drives this effort. I have an important suggestion here: INTEGRATE OFTEN such as once an every week or every two weeks. Because other wise it will be hard to keep in line regarding the major changes during development.

Pros:
- Gives you the ability to trace undesired affects of the integrated modules on each other, before releasing an alpha version of your product.
- Good approach for locating hard to find bugs.
- Confirms that specifications and protocols are doing fine in coordination.

Cons:
- Takes more time than a unit testing such as minutes or more depending on your environment
- Requires you to provide a real test bed with real software and hardware units.
- Harder to automate compared to Unit Tests because there may be many dependencies from outer world such as (Dbs, Network conditions etc....)


3-) Black Box Testing: (AXE)

This is actually a very large topic including sub headers such as Regression, Functional, Stress etc. tests . So from a developer point of view this effort is driven by other people, tools or organizations such as Manual and/or Automated Test Engineers, 3rd party testing organizations, Black Box Automated Test Tools etc. So I will leave you the googling part and just mention about main Pros and Cons.

Pros:
- Overall software product is tested by professional testers in alpha and/or beta cycles before releasing the product to the market.
- In this cycle inconsistencies are diminished from a user point of view.

Cons:
- You really need a good test bed, test setup, automated test environment for black box testing.
- Takes days or even months to test the overall product.

KILLING THE MOSQUITO SCENARIO

- You use the AXE to smash the mosquito after many tries you are very tired, the walls are ruined and the mosquito is still alive and kicking.
- Than you decide to use the ELECTRICAL BURNER, you wait for the sun to settle down and the room is dark you plugin the device and wait for the mosquito, but the mosquito found another light source and made his way along that direction and he never hit the electrical burner. You are very tired and sleepless, the mosquito is still alive and buzzy.
- Than next morning you decide to smash the mosquito on the wall, you are very tired so after many attempts you finally managed to get a bloody spot on the wall.

That mosquito was a NULL POINTER that some how was never hit during your Integration or Black Box tests.


Have a nice day;

Baris ERGUN.