Tools & Dev Ops & Middleware

1. ivy vs maven
Ivy is dedicated as a repository for dependency hosting/management. Ivy is normally working with ant.
Maven is more than dependency management, however, has become one of the most popular dependency repository.

2. make vs ant vs maven vs gradle

make is the dinosaur age build tool, was since 1960s.
ant is also dedicated build tool, which create “target” to run using the ant libraries (java).
maven has another hat of being a build tool. it starts with settings.xml(the repository locations) and pom.xml (the individual configuration for project).
one advantages maven over ant is, maven has defined a lot conventions (which become kind of standard and provides a lot convenience, similar to spring’s being “opinionated”.) so instead of telling ant, where is the class & resources to compile from and into, maven comes with default compile command which works unless you have different than convention folder structure, (which then can be simply configured in pom.xml to tell maven).

+---src
|   +---main
|   |   +---java
|   |   |   \---com
|   |   |       \---best2lwjj
|   |   |           \---services
|   |   |                   Super.java
|   |   |                   
|   |   \---resources
|   \---test
|       +---java
|       \---resources

gradle a new challenger, which features groovy scripts instead of xml. (build.gradle & settings.gradle) which provides “unlimited” functionality conveniently. (instead of build a library or maven plugin, build.gradle can be written using a full functioning groovy language.)

3. CD & CI: jenkins vs hudson
both actually come from same origin. Hudson is first started by SUN (before being acquired by Oracle many years back). It was open source before.
After oracle bought over, the open source community since has moved to create Jenkins from the same original source code. (Jenkins become way more popular now.)

both and same as other CI & CD tools basically polling the source code repo (being svn, cvs or git); then take corresponding actions (configurable), such as ant compile/maven build/gradle integration testing etc….

4. CI & CD with gitlab
git become more and more popular. (git being another project from the Linus Torvalds, has borrowed a lot, like file system from Linux.)

one difference from other CI tools, gitlab enables customized gitlab runner, which is a dedicated server/servers to run certain tasks (configurable through tags). this creates a lot possibilities.
for example, one thing i have done in goldman was, to create a new pipeline, which was able to pick up code changes from feature branch, compile, test, integration test, build, package, put onto cloud/repository, deploy it and restart the server. all in one go, without single manual intervention.
This become possible with the gitlab tags and runners.

5. gitlab
normally master/ rc-xx / release-xx are feature branches, which are protected and monitored for automations/CI/CDs.

6. common issue with maven dependencies

https://stackoverflow.com/questions/4701532/force-maven-update/9697970#9697970

Front End

1. redux flux

flux-528x174

Flux:
flux

redux
redux.png

2. jsx virtual dom
20160604082917065

3. reactJs redux
20180530214840982

basically similar to the flux flow(redux is one flux implementation), in (react) redux, component (user action, for example click a button) would trigger/generate an action, the action are being wrapped(like a container) within redux, it has a list registered listener (reducer) would act on the action (in redux, action is a noun, like a domain object, to tell the type of the action, and any additional parameters), reducer could 1) construct a new state (from existing + the new action), then store them in the store; 2) or with thunk, if could really take some action, for example, call a rest service, then generate a new state
after the new state, (which is always maintained/persisted in the store), the interested component could listen (observer) to these state (mapStateToProps), then update the component (display) correspondingly.

20180530223517561

20151210234529139

4. set up store, reducer, thunk

const store = createStore(
  reducer,
  applyMiddleware(thunk)
);

5. thunk
basically, “plain” redux, (supposed to be pure), only take in plain action (as a domain object, POJO); however, there are cases, it requires to run certain actions, (calling web service), with thunk, the “action” could be a function which then return a action(the real pojo).

export function getPosts_actionCreator_Thunk() {
  return function(dispatch) {
    return fetch("https://lwpro2.wordpress.com/super/posts")
      .then(response => response.json())
      .then(json => {
        dispatch({ type: "POST_LOADED", payload: json });
      });
  };
}

6. saga
sage is another way to implement what thunk’s. to iterate, Redux does not understand other types of action than a plain object.(for redux, actions is plain domain object).
Thunk is a middleware to extend redux, so that a function (thunk function) is put into the action creator, which could be run, and then return the action (domain object).
While saga is working in another approach. Saga is like an interceptor. So the original action creator is same, which simply return a plain domain action. However, saga can intercept all actions, if it’s saga’s interested actions (listener/observer pattern), it then intercept into its logic (the call of web service for example).

export function getPosts_actionCreator_original() {
  return { type: "POST_LOADED" };
}

export function loadPost_realFunction_original() {
    return fetch("https://lwpro2.wordpress.com/super/posts")
      .then(response => response.json());
}

import { takeEvery, call, put } from "redux-saga/effects";
export default function* interceptor_observer_Saga() {
  yield takeEvery("POST_LOADED", workerSaga);
}

function* workerSaga() {
  try {
    const payload = yield call(loadPost_realFunction_original);
    yield put({ type: "POST_LOADED", payload });
  } catch (e) {
    yield put({ type: "POST_LOAD_ERROR", payload: e });
  }
}

frameworks

1. Spring DI IoC
without frameworks, normally to invoke methods on another class (loose coupling), we need to create an instance of the class, then call the method.
IoC, inversion of control, is pre set up this for us, so called inversion of control. instead we call that method on that object, the object is prepared and injected for us.

DI is one implementation of IoC. Both spring and Guice are DI frameworks.
This saves a lot of efforts for developers, the cost is slower app start up time (so the effort/time is brought forward to spend). This is initial reason why spring started becoming popular even during webwork/struts days.

2. @autowired
spring used to use XML for bean creation, and DI. basically tell what’s the beans/class/Objects to be managed by spring bean factory/context. and what’s needed fields/beans for the beans to be created. Spring library would read these configuration and do the set up job.

with the annotation (using reflection) becomes popular, this saves efforts to maintain separate XML files, (at the same time, become less maintainable). So the XML configuration start having the annotation equivalent. @Auowired is the equivalent of telling needed constructs for bean creations in XML.

3. Spring MVC
this is the spring implementation of MVC pattern, corresponding to structs and java default servlet and other framework. basically to tell what’s the URL mapped to which class, and what’s the result(view), with the model passing around. (in XML)

with annotation this become, @RestController @Controller @RequestMapping @RequestParamter etc

4. thread safety in Spring
by default, spring beans are singleton, though this can be changed to be prototype (one bean per each request)

singleton beans are not thread safe, same as servlet, which is shared among requests/threads.

5. spring AOP
AOP is for those code cross cutting or scattering. there are a lot common tasks, like audit or access control, which without AOP, could be duplicating the same code in multiple different places
AOP basically define the point cut (places), cross point and advice (the real business to be done, like audit /access control)
Spring has mostly two proxy mechanism for AOP, default JDK dhynamic proxy or CGLIB (JDK is preferred by spring for performance consideration)

6. spring boot
spring boot trying to make the developers’ job easier, by providing a lot defaults (configuration over convention)
instead of XML, or a lot annotations, Spring boot assume conventions (and library on classpath)to preset up

7. hibernate
new Java EE ORM implemntation (no more EJBs), JPA, has same origin as new hibernate implementation. so there are a lot commonality.
Hibernate probably is the most popular JPA provider for now

8. myBatis
While hibernate translate between objects and SQLs, where we call objects create/insert/update/delete actually call the hbernate generated SQL to be run on RDBMS (or other DB);
myBatis is more for the reverse. if the DB is old, not possible to change schema, direct SQL is preferred than leaving to Hibernate to construct simple queries.
Even though hibernate has its own HQL, mybatis is since beginning more on the direct SQL type

9. apache camel vs spring integration
both are EIP. Apache Camel has a massive of supported start points/URI, like HTTP, MQ, Timer etc.
With Spring framework become more and more popular, it started to incorporate various other functionality (besides DI), Spring integration is Spring’s counterparts for Camel.

JVM

  • 1. reflection
  • java reflection is to get and modify existing class, method, field behavior, like access modifier.
    To note, this is an expensive operation.

  • 2. serialization
  • Serializable is a marker interface. ObjectOutputStream’s writeObject method is for serialization and ObjectInputStream’s readObject is for deserialization. These methods could be override to provide customized serialization.

    Transient variable will be ignored during serialization.

  • 3. dynamic proxy
  • this follows the proxy pattern, kind of adding a proxy/facade before the read method invocation.

  • 4. clone
  • default clone method (native method in Object) is shallow copy; which is primitive types are cloned with value; objects are
    “cloned with value (memory address)”

    to do deep clone, serialization is one approach. another approach is to override the clone method for object variables.

  • 5. memory
  • stack is for methods runs, each method corresponding to one stack frame added ontop of each other, and removed once the method finish;
    local primitive methods during the method invocation are stored on stack, and cleared once method finish; while object created/referred in methods are put onto heap

    Stack can be set up during JVM start up using -Xss (maximum for stack memory)
    one example for possible stack overflow is, for recursive function calls, if it went too deep, it could cause stackOverFlow

    heap is the main memory portion within JVM, can be divided into two parts Yong Generation (Eden + S1 + S2/Survivor) and Old Generation/Tenured space

    Before Java 8, it’s using PermGen to store class info, metadata, and string pools. After java 8, permGen is removed, and something equivalent is created called MetaData space

    recent JVM implementation are using generational GC. which is minor GC on young generation (where there is not enough eden space to put new objects); major GC on tenured space (when there is not enough memory on Tenured space to put objects)

    There are multiple GC algorithms, like serial, paralel, ParNew, CMS, G1 etc. for young generation, most recent JVM (hotspot, IBM, JRocket) are using copying algo for GC on young generation, because more of the objects are actually not long lasting. So after objects created and put onto Eden, the first Young GC could have most of the objects cleared. while remaining (minority) survived would be COPIED to S1/To Survival space. At any time, there is always one S1/S2 survival space empty to be copied to.
    by default after 15 cycles (this can be changed by setting up the survival ratio as JVM parameter), (if the object has been surviving after 15 coping GC), it will then be promoted to tenured space.
    There are exceptions, if the object is too big to put onto eden of Survival space (eden to S1 to S2 is 8:1:1), it could promoted to Tenured space immediately.

    for GC on tenured space, before Java 8, for recent JVM, it’s using CMS. (which is good for concurrency, and first mark the object to be cleared, which then invokes the finalize method, for last try, if the object still not referenced (used to be using reference count, now new JVM using GC roots reach-ability), then the object would be swept.)

    note: JVM stack is for the stack for Java methods/stack; while native stack is for native (non-java) methods.

    0d5d2f77322891081e879206425b1b37

  • 6. class loading
  • in Java, it’s delegating Parent class loader to load the class. so unless the class cannot be loaded from bootStrap (JRM/lib), Extension (JVM/lib/ext), System class loader, and the parent class loader(could be EAR, then WAR, then JAR), then child class loader.

    20180130145726613
    this implementation would avoid the classCast exception, where different copies of same class loaded by different class loader, which is incompatible with each other.

    20180130115349725

  • 7. memory leak
  • There are two types of memory leak. if objects are created, and later no longer used, however cannot be GCed, these objects would continue to hog the memory. examples are, the objects are referred by static variables of some long lived classes (class are normally live in permGen/Meta space, and would live till JVM exits); or the objects are referred by collections (the object might no longer used, however, the collection still hold a pointer to the object); inappropriate implementation of the object, for example equals & hashCode. it could happens the object could be put on a hash bin/node/bucket, however, never able to be retrieved, as the equals method doesn’t comply with hashCode implementaion.

    another type of memory leakage, is object creation faster than GC. there are some framewoks/library/or wrong implementations, generating objects (per thread, per request, as proxy, as prototype) very fast, which GC is not able to catch up, could cause OOM.

  • 8. happens before
  • JVM could re-arranged order of code executions for better performance (JIT compiler optimization, for example grouping operations on same object together). however, for certain operations, the happens before must satisfy no matter how the code re-arranged.

    happens before, means the previous operation visible to following operation.

    volatile: write operation must happens before for the read operation. since volatile variables are directly operating on the main memory, so write of the variable would be visible for all subsequent reads.

    lock: unlock happens before unlock. only once the lock released (marker work removed the thread info), subsequent calls can then acquire the lock (put the new thread’s info onto the marker word).

    Transitivity: if A happens before B, B happens before C, then A happens before C.

    Thread start rule: the start method happens before every actions on the spawned thread.

  • 9. permgen vs metaspace
  • Before Java 8, it’s using PermGen (Permaneent Generation, the JVM Generational) to store those class, meta data info. PermGen was part of the JVM. so if too many classes initiated for example, it could cause the permGen space generate OOM. (String pools were also put into permGen before for some old HotSpot JVM, before Java 7.)

    ldFRR

    eP0SJ

    from Java 8, permGen has been removed, and it has allocate a new space called MateSpace (kind of like renaming) for storing same information. However, metaSpace now lives on the native memory, which literately wont throw OOM. (unless whole native memory for the JVM process used up, in which case OOM wont be thrown either as the JVM is killed.)

    JVM Default maximum PermGen size (MB) Default maximum Metaspace size
    32-bit client JVM 64 unlimited
    32-bit server JVM 64 unlimited
    64-bit JVM 82 unlimited

    permGen size could be adjusted same as other vm paramters, (like oss, xms, xmx. etc )-XX:PermSize=N, or -XX:MaxPermSize=N.
    note, if the permGen is set to small initially, it could require multiple increases (before reaching maxPermSize). each resizing would cause a FULL GC.
    This could happens during JVM start up, as it realizes more permGen (or xms, heap space) needed, it would busy resizing, which take/waste a lot time.
    so one important JVM tuning is to set up a proper permSize (same as xms), so that resizing, and full GC (expensive operations) could be minimized.

    same as permGene,
    -XX:MetaspaceSize –the initial of the Metaspace

    -XX:MaxMetaspaceSize –the maximum size of the Metaspace

    7d0Nu

  • 10. Java Memory Model (JMM)
  • JMM

    JMM_generation

    JMM_gc

    JMM_localCache

    JVM

  • 11. garbage collection (GC)
  • right now, most JVM (HotSpot, JRocket, etc) are using generational JVM memory modeling and GC.
    This is the default space allocation on heap.

    JVM_Generation

    if not enough space on new space (object too big to put onto eden), it could trigger minor GC. because most of the object are short lived, so minor GC is using copy algorithm, basically remove most objects (since most are short lived), and copy surviving objects into one of the survivor space (from/to , or s1/s2). default survival ration is 15. once the object has passed 15 minior gc and survived, it will be promoted to tenured space (like tenured). another case for promotion, is if the object is too big (even larger than S1/S2 space for example, so no way for them to stay on new space). (“too big too fail/to kill early”)

    on Tenure space, before java 8/9 (using G1), CMS is most popular. basically it’s concurrent version of sweep. if an object (GC roots not reachable), it would be marked (run the finalize method), still not reachable, then swept.

  • 12. program registers
  • PC (program counter register) is thread bound (live per thread), used for register current execution code line number, and direct for next running line (number).
    if it’s running java method, registered is the line number of the byte code.
    if it’s running native method, register is undefined.