when annotating an advice method with the pointcut, spring throws an error

error Type referred to is not an annotation type: LogCommands

Turns out spring is picking up the annotation name and using it for reflection and then load the interface for AOP stuff.

So the solution is to mark the type as complete full interface name, (unless its within same package)

for example, instead of

    @Before("@annotation(LogCommands)")
    public void logCommands(JoinPoint joinPoint) throws Throwable {
        String command = joinPoint.getSignature().getName();
        String input = Arrays.stream(joinPoint.getArgs()).toString();
         .....
    }

a FQN make it work

@Before("@annotation(com.abc.svc.annotation.LogCommands)")
public void logCommands(JoinPoint joinPoint) throws Throwable {
    String command = joinPoint.getSignature().getName();
    String input = Arrays.stream(joinPoint.getArgs()).toString();
    .....
}

the default `gradlew bootRun` command is not always working out. The latest error message I got is

as an alternative, the plain build works, even though with a caveat:

caveat: the built out package has been put under libs folder, instead of what the bootScripts is expecting (lib). So point to the right path will work.

PS C:\Users\...> .\build\bootScripts\cli.bat
Error: Unable to access jarfile C:\Users\...\build\bootScripts\..\lib\cli-0.0.1-SNAPSHOT.jar

PS C:\Users\...\build> java -jar .\libs\cli-0.0.1-SNAPSHOT.jar



2020-08-23 10:08:55.583  INFO 53436 --- [           main] : Starting Application on ....
2020-08-23 10:08:56.493  INFO 53436 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2020-08-23 10:08:56.580  INFO 53436 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 72ms. Found 3 JPA repository interfaces.
2020-08-23 10:08:57.191  INFO 53436 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration' of type [org.springframework.transaction.annotation.ProxyTransactionManagementConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
2020-08-23 10:08:57.823  INFO 53436 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2020-08-23 10:08:57.864  INFO 53436 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]

kubectl get po -A | grep Terminating | awk '{ system("kubectl delete po " $2 " -n " $1) }'

force delete

kubectl get po -A | grep CrashLoopBackOff | awk '{ system("kubectl delete --grace-period=0 --force po " $2 " -n " $1) }'

all non-running state

kubectl get pods --field-selector=status.phase!=Running -A | awk '{ system("kubectl delete --grace-period=0 --force po " $2 " -n " $1) }'

an alternative with splice

  
myServiceList.splice(
    0,
    myServiceList.length,
    ...myServiceList.filter((value) => value.name && value.name.length > 0)
  );

import functools

def wrap(name: str):
    def wrapper(f):
        print("within the first wrapper")
        print(f, name)

        @functools.wraps(f)
        def wrapper2(place: str):
            print("this is the place", place)

            result = f(place)
            print(result)

        return wrapper2

    return wrapper

@wrap("tester")
def handle(place):
    print("@", place)
    return "@"+place



result = handle("Singapore")
print(result)

The M.2 SSD didn’t come with a screw. So this is what I did, being an engineer:

port the screw over, and built a washer:

it’s not unusual that there are cases we have new graphql servers to be added, or existing graphql servers updated.

Apollo server at the moment, have a gateway component which could statically maintain and route the right traffic to the right server and aggregate results.

However, this community version of Apollo server and gateway is not able to cater for the case in the beginning, where we would like to hot update/reload the gateway with any updates however with shut down the servers.

After some trials and errors, this is the work around to sort this out:

instead of hardcode the service list (apollo graphql servers), maintain it separately and dynamically:

#### instead of this
const gateway = new ApolloGateway({
  serviceList: [
    { name: "astronauts", url: "http://localhost:4002" },
    { name: "missions", url: "http://localhost:4003" }
  ]
});

## maintain this list
#### instead of this
const dynamicServiceList = [
    { name: "astronauts", url: "http://localhost:4002" },
    { name: "missions", url: "http://localhost:4003" }
  ];

then from apollo gateway, instead of spoon feeding the list, switch it to dynamically load the definitions

const gateway = new ApolloGateway({
  serviceList: [],
...
  experimental_updateServiceDefinitions: async (serviceList) => {
    return {
      isNewSchema: true,
      serviceDefinitions:  await Promise.all(dynamicServiceList.map(async (service) => {
//load the sql
        const data =  await request(service.url, sdlGQL);
...
        return {
...
//then feed the data here
          name: service.name,
          url: service.url,
          typeDefs: parse(data._service.sdl)
        };
      }))
    };
  }
});

At the same time, create a new endpoint, if needed, to take in updated servers, or new servers, so that to update the dynamic list

app.get('/hot_reload_schema/', async (req, res) => {
//get the new server info from req
....
      const status = await validateSDL(name, url);
      if (status){
//update the dynamic list
        dynamicServiceList.push({name, url});
        res.send('The new schema has been successfully registered');
      }
      else {
        res.send('Please check the log for the error details while registering this schema');
      }
    }
);

lastly, make sure the gateway is polling the service list at some intervals:

new ApolloGateway({
  pollingTimer: 100_000,
  experimental_pollInterval: 100_000,
....
});

alternatively, this could also be updated by an explicit load:

new ApolloGateway({
....
}).load();

have contributed the solution to the community here.

lua-resty-openidc is a certified OIDC and OAuth library built onto openresty. While openresty is a reverse proxy built on nginx with lua and luaJit embedded, which greatly upgrade nginx’s capability.

lua-resty-openidc is able to authenticate and authorize the client with compliant OP (keycloak in my case). However, I was facing issues with infinite redirects:

    location /test {

      access_by_lua_block {

        local opts = {
          discovery = "http://keycloak/...../.well-known/openid-configuration",
          redirect_uri_path = "/test",
          accept_none_alg = true,
          client_id = "xxxx",
          client_secret = "xxxxx",
          use_nonce = true,
          revoke_tokens_on_logout = true,
        }

        local res, err, url, session = require("resty.openidc").authenticate(opts)

        if err or not res then
        ngx.status = 403
        ngx.say(err and err or "no access_token provided")
        ngx.exit(ngx.HTTP_FORBIDDEN)
        end
      }
      default_type text/html;
      content_by_lua 'ngx.say("<p>hello, world here from test</p>")';
    }

for above block, I was expecting the library able to direct the client to keycloak authentication at first time, then subsequently redirect back to the redirect_uri /test, which then see the client is already authenticated, and proceed to the content_by_lua` block.

however, instead, it’s facing a infinite redirect between keycloak and redirect_url: https://github.com/zmartzone/lua-resty-openidc/issues/32#issuecomment-656035986

the final solution is to put the control block (access_by_lua) after location /, then worked out

===================================================

a follow up to the original post, the redirect_uri itself could be causing the issue. Instead of pointing it to a final landing page, point it to a intermittent place which would then be directed to the original place (the protected location) should sort the problem as well.

https://github.com/zmartzone/lua-resty-openidc/issues/343

I was facing some issue when embedding the code into google site:

turns out google was possibly running some checks, maybe the traffic while the code is added, and with some activities from any add-on or chrome itself, google site think it’s suspicious and blocked it.

the solution was to run chrome in incongnito mode, it will work fine.

It was unintuitive and the nginx doc doesn’t put it clearly, http://nginx.org/en/docs/http/ngx_http_core_module.html#location.

for nested blocks, for example

    location /protected {
      access_by_lua_block {
        local opts = {
          redirect_uri = "/auth_redirect",
...
          use_nonce = true
        }
...
      }
      default_type text/html;
      content_by_lua 'ngx.say("<p>Protected</p>")';


      location /protected/api {
          ...
      }

      location /auth {
          ...
      }
...
}

the URL, http://{hostname}/protected/auth wont work

while at the same time, the URL http://{hostname}/protected/api would direct correctly.

as the nested location is still following the same pattern or prefix matching, without any url concatenation.

Problem:

There is this weird message on chrome, saying

and at chrome://management, it says

turns out, this is due to some deprecated extensions.

Solutions:

To sort out the issue, on Windows, go to regedit, then search for any suspicious chrome registry:

HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Chrome
HKEY_CURRENT_USER\SOFTWARE\Policies\Google\Chrome

for my computer, there is this deprecated EnablePluginPolicy, and with registry of AliSSO and AliWangwang.

After clearing these registry, and restarted chrome. The weird warnings are all cleared.

chrome://policy/ is clean

another option to verify the PVC mount is to run the df command from the container

have been facing various different issues with one EKS cluster,

• the pods is stuck at failedMount, even though the PV and PVC has been in bound state, and the PVC also shows bound to the specific pods, the EFS on aws is also in good state

## relevant error message
list of unattached/unmounted volumes

Warning  FailedMount            1m (x3 over 6m)  kubelet, .....  Unable to mount volumes for pod "mypod1_test(c038c571-00ca-11e8-b696-0ee5f3530ee0)": timeout expired waiting for volumes to attach/mount for pod "test"/"mypod1". list of unattached/unmounted volumes=[aws]


Normal   Scheduled    5m45s                default-scheduler                                       Successfully assigned /test3 to ....
  Warning  FailedMount  83s (x2 over 3m42s)  kubelet, ...  Unable to mount volumes for pod "test3_....(fe03d56e-aa26-11ea-9d9c-067c9e734f0a)": timeout expired waiting for volumes to attach or mount for pod ""/"test3". list of unmounted volumes=[mypd]. list of unattached volumes=[mypd default-token-mznhh]
 

• related, it shows the efs csi driver not available

aws ebs csi driver 

Warning  FailedMount  9m1s (x4 over 15m)     kubelet, ....Unable to mount volumes for pod "test3_...(f0d65986-aa27-11ea-a7b2-022dd8ed078a)": timeout expired waiting for volumes to attach or mount for pod ""/"test3". list of unmounted volumes=[mypd]. list of unattached volumes=[mypd default-token-mznhh]
  Warning  FailedMount  2m35s (x6 over 2m51s)  kubelet, ... MountVolume.MountDevice failed for volume "scratch-pv" : driver name efs.csi.aws.com not found in the list of registered CSI drivers
  Warning  FailedMount  2m19s                  kubelet, ....  MountVolume.SetUp failed for volume ".." : rpc error: code = Internal desc = Could not mount "fs-43b99802:/" at "/var/lib/kubelet/pods/f0d65986-aa27-11ea-a7b2-022dd8ed078a/volumes/kubernetes.io~csi/scratch-pv/mount": mount failed: exit status 1
Mounting command: mount
Mounting arguments: -t efs fs-43b99802:/ /var/lib/kubelet/pods/f0d65986-aa27-11ea-a7b2-022dd8ed078a/volumes/kubernetes.io~csi/scratch-pv/mount
Output: Failed to resolve "fs-43b99802....amazonaws.com" - check that your file system ID is correct.
See https://docs.aws.amazon.com/console/efs/mount-dns-name for more detail.

• pods stuck at termination state, even thought the svc, deployment, and rs has been killed, the pod still has been stuck at termination

the final solution to sort out all is to reboot the EKS worker node

Have been continuous receiving this error message during the `helm upgrade` even with several retry

UPGRADE FAILED
Error: kind StorageClass with the name "..." already exists in the cluster and wasn't defined in the previous release. Before upgrading, please either delete the resource from the cluster or remove it from the chart
Error: UPGRADE FAILED: kind StorageClass with the name ".." already exists in the cluster and wasn't defined in the previous release. Before upgrading, please either delete the resource from the cluster or remove it from the chart

Have tried two solutions, (the 1st doesn’t work out)

1. as the helm upgrade error message, I did a helm delete the storageclass, then run the helm upgrade, even though on the internet some claim this work, but actually no, it fails again with same error message
2. did a helm rollback to a previous working release, then run the helm upgrade again, then worked

helm history <release>
helm rollback <release> <a-previous-working-revision>
helm upgrade --install <release> .....

At the same time, for helm 2, all helm release information are stored in configmap

## use this command to get the release info
kubectl get  configmaps -n kube-system <release> -o jsonpath='{.data.release}' | base64 -d | gzip -cd

as for helm 3, it’s stored in secrets.

Microsoft Edge/Opera/Google Chrome has become the de-facto browser for internet. This suite here will greatly further enhance your browsing experience:

Problem statement:

we tend to have so many tabs open, it becomes difficult to find the specific tabs among so many we have opened.

Instead, we normally tend to just open a new tab which has already been open and hidden somewhere, this will additionally consume more of our computer memory usage.

We have visited certain pages before, but really cannot recall the complete address of the page.

We are trying to search for a term, however, would like to see only the top suggestions.

We might have different search engines would like to leverage on, for example maybe bing for desktop images, or baidu for chinese languages.

We have a limited screen, would like to maximize the space screen, instead of leaving the address and tab to consume the precious screen space.

Solution:

Install the Tab & URL manager here, 
https://chrome.google.com/webstore/detail/tab-url-manager/egiemoacchfofdhhlfhkdcacgaopncmi

Use the shortcut to open the Tab & URL manager
Windows: Alt+L

Mac: ⌥+L

In addition, you can configure further keyword mapping in the Extension Options.

* type a URL followed by enter to visit the specified address
* type a term followed by enter to search the term (the search engine is configurable)
* type a term to navigate to any popular search suggestions
* type a keyword to navigate to any mapped URL (the map is configurable)
* type a term to navigate to any open tabs
* type a term to get the likely result you would like to visit

ShortCut for URL mapper

Problem Statement:

It’s always difficult to remember the long URLs. Yet, we could have so many pages we normally visit.

Solution:

Install the ShortCut for URL mapper here, 

https://chrome.google.com/webstore/detail/short-cut-for-url-mapper/lafchflokhmpcoaondfeffplkdnoaelh

Click the extension, to configure the keyword to URL mappings.

After that, just type sc in the address bar, followed by the keyword would bring you to the targeted page.

My Tabs

Problem Statement:

We all have list of often visited pages each. Normally we would either bookmark them, then click one by one to open each, or just open a new tab each and type the URLs manually.

This is very tedious and waste our precious time.

Solution:

Install MyTabs here
https://chrome.google.com/webstore/detail/mytabs/mjapfgokeoeigopkkjnlkhgpcgoaoenf
Click the extension, to configure the often visited pages.

Then each time auto load them all with one click Alt+Shift+T, or if using Mac, Option+Shift+T.

I have been scratching my head trying to understand how nginx is able to log to console without any manual intervention.

turns out, in the dockfiler, they have created a symbolic link

https://github.com/nginxinc/docker-nginx/blob/8921999083def7ba43a06fabd5f80e4406651353/mainline/jessie/Dockerfile#L21-L23

Recently, I have set up my micro services application onto datadog APM with tracing and logging.

something like,

so that we can follow through, for example, any http request or user_id from HTTP request, controller, service, authentication, database, and etc, with the same tracing ID and logs detailed.

Here are the steps I have followed:

1. Set up datadog account and access, so that we will get the datadog API key, which is to be used to upload the data to datadog server
2. install the datadog agent onto the server, we are using kubernetes on AWS (EKS) here, and have installed it as a daemonset (so that we have one for each node) with terraform.

set up the necessary environment variable in the daemonset

and expose the necessary ports

3. from the python services, leverage on libraries, for example, https://pypi.org/project/JSON-log-formatter/0.1.0/, to log the messages into JSON format

format = "%(asctime)s %(levelname)s [%(name)s]-- '[dd.trace_id=%(dd.trace_id)s dd.span_id=%(dd.span_id)s] ' -- %(message)s"
json = jsonlogger.JsonFormatter(format)

handler = logging.StreamHandler()
handler.setFormatter(json)

so that these logs would be in JSON format, and datadog would be able to parse the logs into tags & attributes.

4. then configure the datadog tracing

from ddtrace import config, patch_all, tracer


def setup_datadog():
    tracer.configure(
        hostname=os.environ["DD_AGENT_HOST"],#set up in the deployment.yaml, the env attribute
    )
    tracer.set_tags({"env": "production"})
    config.flask["service_name"] = "checkout-service" #this could also set up in the deployment.yaml, the env attribute
    config.flask["analytics_enabled"] = True
    config.flask["extra_error_codes"] = [401, 403]
    patch_all()

5. finally, then annotate the entry point function, with

@tracer.wrap(service="checkout-service")

sample configuration in the deployment.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
...
  annotations:
    ad.datadoghq.com/checkout.logs: '[{"source":"python", "service": "checkout-service"}]'
spec:
  replicas: 1
...
  template:
    metadata:
    spec:
      containers:
      -  name: checkout
...
        env:
          - name: DD_AGENT_HOST
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP
          - name: DD_SERVICE
            value: checkout-service
          - name: DD_TRACE_ANALYTICS_ENABLED
            value: "true"
...
      restartPolicy: Always
status: {}

Sometimes, PV and PVC could stuck at terminating. This is because there is a finalizer to protect the PV and PVC termination while there are still possible usage.

For example, when I am trying to delete the PV

both PV and PVC however are stuck for quite a while

the resolution is to edit and remove the finalizers:

then without the finalizers, it will let the PV and PVC termination.

endpoints is normally used behind the scene, without the need of any manual intervention.

However, for local testing, it could become helpful to leverage on the endpoints customization.

---	
  kind: "Service"
  apiVersion: "v1"
  metadata:
    name: "svc-to-external-web"
  spec:
    ports:
      - name: "apache"
        protocol: "TCP"
        port: 80
        targetPort: 80 
---
  kind: "Endpoints"
  apiVersion: "v1"
  metadata:
    name: "svc-to-external-web" 
  subsets: 
    - addresses:
        - ip: "8.8.8.8" #The IP Address of the external web server
        ports:
        - port: 80 
          name: "apache"

for example, with above, if we create a service without any selector (of any pods/deployments), and manually create an endpoint with the same name as the service, then we can configure where the endpoint point to, which could be an external IP address, or maybe to a docker service

like

apiVersion: v1
kind: Endpoints
metadata:
  name: svc-to-external-web
subsets:
- addresses:
  - ip: 192.168.64.1 # minikube ssh "route -n | grep ^0.0.0.0 | awk '{ print \$2 }'"
  ports:
  - port: 80

ref: https://theithollow.com/2019/02/04/kubernetes-endpoints/

i have created a custom scheduler, which has been in running state.

however, when i create a new pod, assigning it to the custom scheduler, through .spec.schedulerName. the pod has been stuck in pending state.

here are the sample configuration

scheduler.yml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-scheduler
    tier: control-plane
  name: my-scheduler
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-scheduler
    - --authentication-kubeconfig=/etc/kubernetes/scheduler.conf
    - --authorization-kubeconfig=/etc/kubernetes/scheduler.conf
    - --bind-address=127.0.0.1
    - --kubeconfig=/etc/kubernetes/scheduler.conf
    - --leader-elect=true
    - --port=10351
    - --scheduler-nam=my-scheduler
    - --secure-port=10359
    image: k8s.gcr.io/kube-scheduler:v1.16.0
    imagePullPolicy: IfNotPresent

pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  schedulerName: my-scheduler
  containers:
  -  image: nginx
     name: nginx

turns out the issue was with the --leader-elect. seems like the scheduler is not able to move forward with the --leader-elect set to true.

when I am trying to run etcdctl in minikube, there has been an exception:

commands:

ETCDCTL_API=3 etcdctl get "" --prefix=true
ETCDCTL_API=3 etcdctl get "" --from-key

Exception

{"level":"warn","ts":"2020-05-01T09:32:07.933Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-ea0c78af-0a9d-4092-8722-75fed707e112/127.0.0.1:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest connection error: connection closed"}
Error: context deadline exceeded

the solution is to provide the needed ca files

ETCDCTL_API=3 etcdctl get "" --prefix=true --cacert=/var/lib/minikube/certs/etcd/ca.crt --cert=/var/lib/minikube/certs/etcd/server.crt --key=/var/lib/minikube/certs/etcd/server.key

the cert can be found by running

kubectl get pod etcd-minikube -n kube-system -o yaml

according to this post, https://devblogs.microsoft.com/typescript/typescript-and-babel-7/, it seems like babel 7 should work with typescript.

however, i have encountered quite a lot surprises during the integration.

here are the changes I have done:

install the typescript preset

yarn add --dev @babel/preset-typescript @babel/preset-env @babel/plugin-proposal-class-properties @babel/plugin-proposal-object-rest-spread

then configure babel.config.json

(most importantly, i have a "macro" plugin, that’s the reason I would like to switch from tsc to babel as i would like to leverage on the macros)

{
  "presets": [
    "@babel/env",
    "@babel/preset-typescript"
  ],
  "plugins": [
    "@babel/proposal-class-properties",
    "@babel/proposal-object-rest-spread",
    "macros"
  ],
  "ignore": [
    "node_modules"
  ]
}

then in package.json, add these scripts

    "build": "babel . -d lib --extensions '.ts,.tsx'",
    "apprun": "babel-node --extensions '.ts,.tsx' index.ts",
    "rundev": "build && nodemon lib/app/app.js",
    "runProd": "build && node lib/app/app.js"

babel-node is working fine.

however, the babel built output seems quite often is not complete.

for once, it complains transpiled graphql_macro_1 is not defined, as this line is missing from the generated javascript file:

const graphql_macro_1 = require("graphql.macro");

then at this moment, it’s complaining this line

as this variable, regeneratorRuntime is called but looks like nowhere it has been defined in the generate javascript file.

this is the complete package.json, with the relevant packages and scripts

{
  "name": "frontend",
  "version": "1.0.0",
  "license": "MIT",
  "dependencies": {
....
    "graphql": "^15.0.0",
    "graphql.macro": "^1.4.2",
...
  },
  "scripts": {
...
    "build": "babel . -d lib --extensions '.ts,.tsx'",
    "apprun": "babel-node --extensions '.ts,.tsx' index.ts",
    "rundev": "build && nodemon lib/app/app.js",
    "runProd": "build && node lib/app/app.js"
  },
  "devDependencies": {
    "@babel/cli": "^7.8.4",
    "@babel/core": "^7.9.0",
    "@babel/node": "^7.8.7",
    "@babel/plugin-proposal-class-properties": "^7.8.3",
    "@babel/plugin-proposal-export-default-from": "^7.8.3",
    "@babel/plugin-proposal-export-namespace-from": "^7.8.3",
    "@babel/plugin-proposal-object-rest-spread": "^7.9.5",
    "@babel/preset-env": "^7.9.5",
    "@babel/preset-flow": "^7.9.0",
    "@babel/preset-typescript": "^7.9.0",
    "babel-watch": "^7.0.0",
...
  }
}

vs https://www.typescriptlang.org/docs/handbook/compiler-options.html, tsc is doing much better job. the only caveat is, it cannot generate the necessary babel macros.

ref:

https://babeljs.io/docs/en/babel-preset-typescript#docsNav

https://iamturns.com/typescript-babel/

the original post from microsoft below

Today we’re excited to announce something special for Babel users.Over a year ago, we set out to find what the biggest difficulties users were running into with TypeScript, and we found that a common theme among Babel users was that trying to get TypeScript set up was just too hard. The reasons often varied, but for a lot of developers, rewiring a build that’s already working can be a daunting task.

Babel is a fantastic tool with a vibrant ecosystem that serves millions of developers by transforming the latest JavaScript features to older runtimes and browsers; but it doesn’t do type-checking, which our team believes can bring that experience to another level. While TypeScript itself can do both, we wanted to make it easier to get that experience without forcing users to switch from Babel.

That’s why over the past year we’ve collaborated with the Babel team, and today we’re happy to jointly announce that Babel 7 now ships with TypeScript support!

How do I use it?

If you’re already using Babel and you’ve never tried TypeScript, now’s your chance because it’s easier than ever. At a minimum, you’ll need to install the TypeScript plugin.

npm install --save-dev @babel/preset-typescript

Though you’ll also probably want to get the other ECMAScript features that TypeScript supports:

npm install --save-dev @babel/preset-typescript @babel/preset-env @babel/plugin-proposal-class-properties @babel/plugin-proposal-object-rest-spread

Make sure your .babelrc has the right presets and plugins:

{
    "presets": [
        "@babel/env",
        "@babel/preset-typescript"
    ],
    "plugins": [
        "@babel/proposal-class-properties",
        "@babel/proposal-object-rest-spread"
    ]
}

For a simple build with @babel/cli, all you need to do is run

babel ./src --out-dir lib --extensions ".ts,.tsx"

Your files should now be built and generated in the lib directory.

To add type-checking with TypeScript, create a tsconfig.json file

{
  "compilerOptions": {
    // Target latest version of ECMAScript.
    "target": "esnext",
    // Search under node_modules for non-relative imports.
    "moduleResolution": "node",
    // Process & infer types from .js files.
    "allowJs": true,
    // Don't emit; allow Babel to transform files.
    "noEmit": true,
    // Enable strictest settings like strictNullChecks & noImplicitAny.
    "strict": true,
    // Disallow features that require cross-file information for emit.
    "isolatedModules": true,
    // Import non-ES modules as default imports.
    "esModuleInterop": true
  },
  "include": [
    "src"
  ]
}

and just run

tsc

and that’s it! tsc will type-check your .ts and .tsx files.

Feel free to add the --watch flag to either tool to get immediate feedback when anything changes. You can see how to set up a more complex build on this sample repository which integrates with tools like Webpack. You can also just play around with the TypeScript preset on Babel’s online REPL.

What does this mean for me?

Using the TypeScript compiler is still the preferred way to build TypeScript. While Babel can take over compiling/transpiling – doing things like erasing your types and rewriting the newest ECMAScript features to work in older runtimes – it doesn’t have type-checking built in, and still requires using TypeScript to accomplish that. So even if Babel builds successfully, you might need to check in with TypeScript to catch type errors. For that reason, we feel tsc and the tools around the compiler pipeline will still give the most integrated and consistent experience for most projects.

So if you’re already using TypeScript, maybe this doesn’t change much for you. But if you’re already using Babel, or interested in the Babel ecosystem, and you want to get the benefits of TypeScript like catching typos, error checking, and the editing experiences you might’ve seen in the likes of Visual Studio and Visual Studio Code, this is for you!

Caveats

As we mentioned above, the first thing users should be aware of is that Babel won’t perform type-checking on TypeScript code; it will only be transforming your code, and it will compile regardless of whether type errors are present. While that means Babel is free from doing things like reading .d.ts files and ensuring your types are compatible, presumably you’ll want some tool to do that, and so you’ll still need TypeScript. This can be done as a separate tsc --watch task in the background, or it can be part of a lint/CI step in your build. Luckily, with the right editor support, you’ll be able to spot most errors before you even save.

Second, there are certain constructs that don’t currently compile in Babel 7. Specifically,

• namespaces
• bracket style type-assertion/cast syntax regardless of when JSX is enabled (i.e. writing <Foo>x won’t work even in .ts files if JSX support is turned on, but you can instead write x as Foo).
• enums that span multiple declarations (i.e. enum merging)
• legacy-style import/export syntax (i.e. import foo = require(...) and export = foo)

These omissions are largely based technical constraints in Babel’s single-file emit architecture. We believe that most users will find this experience to be totally acceptable. To make sure that TypeScript can call out some of these omissions, you should ensure that TypeScript uses the --isolatedModules flag.

What next?

You can read up on the details from the Babel side on their release blog post. We’re happy that we’ve had the chance to collaborate with folks on the Babel team like Henry Zhu, Andrew Levine, Logan Smyth, Daniel Tschinder, James Henry, Diogo Franco, Ivan Babak, Nicolò Ribaudo, Brian Ng, and Vladimir Kurchatkin. We even had the opportunity to speed up Babylon, Babel’s parser, and helped align with James Henry’s work on typescript-eslint-parser which now powers Prettier’s TypeScript support. If we missed you, we’re sorry but we’re grateful and we appreciate all the help people collectively put in!

Our team will be contributing to future updates in the TypeScript plugin, and we look forward to bringing a great experience to all TypeScript users. Going forward, we’d love to hear your feedback about this new TypeScript support in Babel, and how we can make it even easier to use. Give us a shout on Twitter at @typescriptlang or in the comments below.

Happy hacking!

I was not able to run my `docker-compose` commands earlier:

for example dcd up

dcd ps

after turning on –verbose, dcd --verbose ps, turns out the issue is with one of the containers:

This is a front end container, which also explains why I am not able to hot reload the pages, even though I already have the volume mount.

and in case, docker kill is also not working, then do a restart of docker, followed by docker stop and docker rm

It’s common that some indirect dependencies are deprecated due to some vulnerability issues. for example,

I have multiple indirect dependencies on `kind-of`

which are being used by various other dependencies (direct and indirect)

which recently found out there is an isuse with `kind-of`

https://github.com/advisories/GHSA-6c8f-qphg-qjgp

and to update this indirect dependencies, the solution is to add a solution block in package.json

after that, then just run `yarn`

it will update the dependencies to correct version

refer to https://classic.yarnpkg.com/en/docs/selective-version-resolutions/

https://itnext.io/fixing-security-vulnerabilities-in-npm-dependencies-in-less-than-3-mins-a53af735261d

There are valid needs to talk to the Kubernetes cluster from segregated docker containers. It’s possible to do so:

Build the pipe from the container to the host machine

There are several ways to connect the host machine. the container is running together with the host, behaving like on the same subnet. you can access it through the public IP.

otherwise, more elegantly, you can leverage on host.docker.internal to talk to the host

Proxy the resources for the Kubernetes cluster

you can start a Kubernetes proxy to talk to the cluster.

kubectl proxy --port=8080 --disable-filter &

Access to the resources

then to talk to the resources in the cluster from the container, you can do

host.docker.internal:8080/api/v1/namespaces/default/services/pod:{port}/proxy/

This is related to, https://lwpro2.dev/2020/04/03/expose-existing-deployment-with-minikube/, as there is an issue with Kubernetes apiserver proxy, https://github.com/kubernetes/kubernetes/issues/89360, which would strip out the parameter for websocket.

The work around is to use minikube tunnel with LoadBalancer, as in above post.

in addition to, https://lwpro2.dev/2020/04/20/git-command-issue-with-openssl/, even though `brew uninstall, brew install` the specific old version of openssl worked for me with git.

however, it still breaks with poetry.

[SSLError]
HTTPSConnectionPool(host='privatepypi', port=443): Max retries exceeded with url: /pypi/table-understanding/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not availa
ble."))

the solution to work out for both is

brew switch openssl 1.0.2s

in addition to, https://lwpro2.dev/2020/04/20/build-a-private-pypi-with-github/, it has took me quite a while to figure out why certain packages i am not able to install using poetry:

Turns out, that’s because PEP has a very weird renaming:

https://www.python.org/dev/peps/pep-0503/#normalized-names

hence, i need to publish my package named as safe-logging, even though the binary is safe_logging:

even though, within the directory, it is named safe_logging

eslint is now the defector linter for front end, even for typescript, especially with the deprecation of tslint.

eslint is able to do both formatting, like line length, trailing semi colon etc; at the same time, it’s able to do syntax checking as well, for example, unused or undefined variables alike.

while at the same time, prettier is really doing a good job for formatting the front end. it’s a very opinionated framework however, with very easy to customize configurations. for example,

{  
"semi": true,  
"trailingComma": "all",  
"singleQuote": true,  
"printWidth": 70
}

with the combination of both, we can leverage on the strength of both, to do lint and format:

npx eslint -c .eslintrc.json **/*.{ts,tsx} --fix ##for format or fix

npx eslint -c .eslintrc.json **/*.{ts,tsx} ## for lint

these are the set ups we need to prepare:

install the dependencies:

npm install -g prettier eslint
## or
yarn add prettier eslint

install the plugins

npm install --save-dev eslint-config-prettier eslint-plugin-prettier
## or
yarn add --dev eslint-config-prettier eslint-plugin-prettier

then configure .eslintrc

{
"root": true,
"parser": "@typescript-eslint/parser",
"plugins": ["@typescript-eslint","prettier"],
"extends": [
"eslint:recommended",
"plugin:@typescript-eslint/eslint-recommended",
"plugin:@typescript-eslint/recommended",
"plugin:prettier/recommended"
],
"rules": {
"@typescript-eslint/interface-name-prefix": 1
}
}

in addition, to use airbnb formatting:

npx install-peerdeps --dev eslint-config-airbnb

then add it into .eslintrc.json

{  
"extends": ["airbnb", "prettier"],  
"plugins": ["prettier"],  
"rules": {    
"prettier/prettier": ["error"]  
}}

recently, I have tried to build a pypi server on top of github.

here are the steps:

1. create a repo, to host those packages
2. at the root level, define the setup.py with the findpackages(), like

import setuptools

with open("README.md", "r") as fh:
   long_description = fh.read()

setuptools.setup(
   name="pypi",
   version="0.0.1",
   author="lwpro2",
   author_email="lwpro2",
   description="A pypi for python packages",
   long_description=long_description,
   long_description_content_type="text/markdown",
   url="https://github.com/lwpro2",
   packages=setuptools.find_packages(),
   classifiers=[
      "Programming Language :: Python :: 3",
      "License :: OSI Approved :: MIT License",
      "Operating System :: OS Independent",
   ]
)

3. at each packages, add the setup.py as well

import setuptools

with open("README.md", "r") as fh:
   long_description = fh.read()

setuptools.setup(
   name="package1",
   version="0.0.2",
   author="lwpro2",
   author_email="lwpro2",
   description="A fake library",
   long_description=long_description,
   long_description_content_type="text/markdown",
   url="https://github.com/lwpro2",
   packages=setuptools.find_packages(),
   classifiers=[
      "Programming Language :: Python :: 3",
      "License :: OSI Approved :: MIT License",
      "Operating System :: OS Independent",
   ]
)

the folder structure will be like

--pypi
------setup.py
------README.md
------package1
-------------setup.py
-------------README.md
-------------__init__.py
-------------functions.py
------package2
-------------setup.py
-------------README.md
-------------__init__.py
-------------functions.py


4. run this command to generate the binaries

python3 setup.py sdist bdist_wheel

the folder will become

--pypi
------setup.py
------README.md
------package1
-------------setup.py
-------------README.md
-------------__init__.py
-------------functions.py
-------------build/
-------------dist/
--------------------package1-0.0.1.tar.gz
------package2
-------------setup.py
-------------README.md
-------------__init__.py
-------------functions.py
-------------build/
-------------dist/
--------------------package2-0.0.1.tar.gz

5. then create a static pypi server which comply to Pep 503, like

6. then add the new pypi server into Pipfile and pyproject.toml

Pipfile

[source]
name = "privatepypi"
url = "http://private/pypi"
verify_ssl = false

[dev-packages]

[packages]
package1 = {version = "*", index = "privatepypi"}

pyproject.toml

[[tool.poetry.source]]
name = "privatepypi"
url = "https://private/pypi"
secondary = true

[tool.poetry.dependencies]
package2= "^0.0.1"

7. then can install packages using the commands as normal

pip install

poetry install

when encountering such issue:

dyld: Library not loaded: /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib

the resolution is to force uninstall and clean install the openssl:

brew uninstall openssl; brew uninstall openssl; brew install https://github.com/tebelorg/Tump/releases/download/v1.0.0/openssl.rb

in addition to https://lwpro2.dev/2020/04/03/expose-existing-deployment-with-minikube/, with kubectl proxy --port=8080 --disable-filter, the container could talk to the kubernetes cluster.

however, if the proxy is not running or not running properly, it could result in two types of exceptions:

Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.8/http/client.py", line 1230, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1276, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1225, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1004, in _send_output
self.send(msg)
File "/usr/local/lib/python3.8/http/client.py", line 944, in send
self.connect()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 184, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 168, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused

alternatively,

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 13463, in list_namespaced_service
(data) = self.list_namespaced_service_with_http_info(namespace, **kwargs) # noqa: E501
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py", line 13551, in list_namespaced_service_with_http_info
return self.api_client.call_api(
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py", line 340, in call_api
return self.__call_api(resource_path, method,
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py", line 172, in __call_api
response_data = self.request(
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/api_client.py", line 362, in request
return self.rest_client.GET(url,
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/rest.py", line 237, in GET
return self.request("GET", url,
File "/usr/local/lib/python3.8/site-packages/kubernetes/client/rest.py", line 231, in request
raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (502)
Reason: Bad Gateway
HTTP response headers: HTTPHeaderDict({'Date': 'Tue, 14 Apr 2020 06:03:12 GMT', 'Content-Length': '0'})

Both $VARIABLE and ${VARIABLE} syntax are supported. Additionally when using the 2.1 file format, it is possible to provide inline default values using typical shell syntax:

• ${VARIABLE:-default} evaluates to default if VARIABLE is unset or empty in the environment.
• ${VARIABLE-default} evaluates to default only if VARIABLE is unset in the environment.

Similarly, the following syntax allows you to specify mandatory variables:

• ${VARIABLE:?err} exits with an error message containing err if VARIABLE is unset or empty in the environment.
• ${VARIABLE?err} exits with an error message containing err if VARIABLE is unset in the environment.

Kubernetes Service vs Deployment

What’s the difference between a Service and a Deployment in Kubernetes?

A deployment is responsible for keeping a set of pods running.

A service is responsible for enabling network access to a set of pods.

We could use a deployment without a service to keep a set of identical pods running in the Kubernetes cluster. The deployment could be scaled up and down and pods could be replicated. Each pod could be accessed individually via direct network requests (rather than abstracting them behind a service), but keeping track of this for a lot of pods is difficult.

We could also use a service without a deployment. We’d need to create each pod individually (rather than “all-at-once” like a deployment). Then our service could route network requests to those pods via selecting them based on their labels.

Services and Deployments are different, but they work together nicely.

source: https://matthewpalmer.net/kubernetes-app-developer/articles/service-kubernetes-example-tutorial.html

check existing venv

pipenv --venv

remove existing venv

<br />pipenv --rm

cd ~/.oh-my-zsh/plugins && mkdir poetry && poetry completions zsh > poetry/_poetry

source ~/.zshrc

When updating the dependency version in pyproject.toml, for example

[tool.poetry.dependencies]
python = “^3.8.1”
streamlit = “^0.51”. ## ==> update to 0.56

If there is already existing a poetry.lock, poetry would throw

(streamlit-base) jackie@jackie streamlit-base % poetry install
Installing dependencies from lock file

[NonExistentKey]
‘Key “hashes” does not exist.’

install [–no-dev] [–dry-run] [-E|–extras EXTRAS] [–develop DEVELOP]

The reason being the hashes in the lock file doesn’t match the content with pyproject.tom.

The way to sort out this is to remove the lock file, then do the install again to generate a new file:

poetry install


ipconfig getifaddr en0


issue with pipenv

(base) ➜ lib git:(develop) ✗ python –version
dyld: Library not loaded: @executable_path/../.Python
Referenced from: /Users/jackie/.local/share/virtualenvs/streamlit-GdDAcdiW/bin/python
Reason: image not found
[1] 81477 abort python –version

(base) ➜ lib git:(develop) ✗ ..
dyld: Library not loaded: @executable_path/../.Python
Referenced from: /Users/jackie/.local/share/virtualenvs/streamlit-GdDAcdiW/bin/python
Reason: image not found

(base) ➜ streamlit git:(develop) ✗ make all-devel

dyld: Library not loaded: @executable_path/../.Python
Referenced from: /Users/jackie/.local/share/virtualenvs/streamlit-GdDAcdiW/bin/python
Reason: image not found

solution:
1. reinstall pipenv

(base) ➜ streamlit git:(develop) ✗ brew uninstall pipenv
Uninstalling /usr/local/Cellar/pipenv/2018.11.26_3… (1,483 files, 21.3MB)

(base) ➜ streamlit git:(develop) ✗ brew install pipenv

1. clear old virtualenv
   
(base) ➜ lib git:(develop) ✗ rm -rf pipenv --venv
(base) ➜ lib git:(develop) ✗ pipenv --venv
No virtualenv has been created for this project yet!
Aborted!
(base) ➜ lib git:(develop) ✗ pipenv --rm
No virtualenv has been created for this project yet!
Aborted!


2. recreate new env with specific version (exist on local OS)
   
(base) ➜ lib git:(develop) ✗ pipenv --python 3.7
Warning: the environment variable LANG is not set!
We recommend setting this in ~/.profile (or equivalent) for proper expected behavior.
Creating a virtualenv for this project…


commands to list current installed node version:

(base) ➜ streamlit git:(develop) ✗ nvm ls
v10.8.0
-> system
default -> 10.8.0 (-> v10.8.0)
node -> stable (-> v10.8.0) (default)
stable -> 10.8 (-> v10.8.0) (default)
iojs -> N/A (default)
unstable -> N/A (default)
lts/* -> lts/erbium (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.17.1 (-> N/A)
lts/carbon -> v8.17.0 (-> N/A)
lts/dubnium -> v10.18.1 (-> N/A)
lts/erbium -> v12.14.1 (-> N/A)

to switch different versions:

(base) ➜ streamlit git:(develop) ✗ nvm use system
Now using system version of node: v12.16.0 (npm v6.13.4)

to set default version

(base) ➜ streamlit git:(develop) ✗ nvm alias default system
default -> system
(base) ➜ streamlit git:(develop) ✗ nvm ls
v10.8.0
-> system
default -> system
node -> stable (-> v10.8.0) (default)
stable -> 10.8 (-> v10.8.0) (default)
iojs -> N/A (default)
unstable -> N/A (default)
lts/* -> lts/erbium (-> N/A)
lts/argon -> v4.9.1 (-> N/A)
lts/boron -> v6.17.1 (-> N/A)
lts/carbon -> v8.17.0 (-> N/A)
lts/dubnium -> v10.18.1 (-> N/A)
lts/erbium -> v12.14.1 (-> N/A)

There are three kinds of ports widely used in kubernetes resource management, for example:
port is the port exposed within the cluster. so other nodes/pods from same cluster can access it through service:port
targetPort is the port exposed from within the pod.
nodePort is the port exposed to outside world. so on public network, it could be accessed as public-ip:nodePort


kind: Service
apiVersion: v1
metadata:
labels:
app: nginx
name: nginx
spec:
ports:
- port: 443
targetPort: 443
- nodePort: 31234
selector:
app: nginx


by default, the logs won’t print out from the kubernetes pod. this is because Kubernetes would buffer the stdin, stdout, stdout.

the way to sort this out is make

PYTHONUNBUFFERED¶

If this is set to a non-empty string it is equivalent to specifying the -u option

in helm chart, something like this

kind: Service
apiVersion: v1
metadata:
  labels:
    app: example
  name: example
spec:
  ports:
  - port: 5000
    targetPort: 5000
  selector:
    app: example
---
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: example
  name: example
spec:
  replicas: 1
  strategy: {}
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: example
    spec:
      containers:
      - image: example
        imagePullPolicy: IfNotPresent
        name: example
        ports:
        - containerPort: 443
        resources: {}
        env:
          - name: PYTHONUNBUFFERED
            value: "0"
      restartPolicy: Always
status: {}

default approach is to use inspect

<pre><code>docker volume inspect VOLUME</code></pre>

while alternatively, we can go into the volume:

<pre><code>docker run --rm -i -v=db-volume:/tmp/volume busybox sh</code></pre>

the -privilege is powerful yet dangerous:

https://blog.trendmicro.com/trendlabs-security-intelligence/why-running-a-privileged-container-in-docker-is-a-bad-idea/

while the alternative is just to mount the sock

docker run -v /var/run/docker.sock:/var/run/docker.sock 

it’s not strictly docker-in-docker, but it should be able to serve most use cases.

keep encountering this error earlier when helm install then failed

Error: could not find a ready tiller pod

the workout is to get the tiller pod:

kubectl get pods -n kube-system

then restart the pod by delete.

minikube ssh

provider “aws” {
region           = “us-east-1”
access_key  = “your-access-key-here”
secret_key   = “your-secret-key-here”
}

or point to the profile

provider “aws” {
region                                  = “us-east-1”
shared_credentials_file  = “~/.aws/credentials”   //default: “~/.aws/credentials”
profile                                  = “tf-admin”                        //default: “default”
}

As the image command is only run during build time, however, while running the container, we might need to access some environment or configuration variable. here is the workaround:

ARG variable=unknown ## Build time
ENV variable=${variable} ## Run time

Then to pass in the arg,

docker build--build-arg version=0.0.1

for docker-compose, then

container:
  image: image
  restart: always
  build:
    context: dockerfile
    args:
      version: ${version} ## alternatively, default a value here

for docker-compose,

${version}

can be used to retrieve the environment variable

<br />set -x

command-to-run

to see all current shell options

set -o

<code>somecommand ${1:-foo}</code>

https://wiki.bash-hackers.org/syntax/pe#use_a_default_value

Docker will copy files into the container with the permissions of their source

hence if the file, for example already has the execute permission, then no need to,

chmod +x

on and off there are needs to find the tag or release containing certain commits:

git tag --contains hash7ab

been running terraform plan on some limited changes on the .tf file.

however, the plan always generate whole project as need to add, even with forced refresh

terraform refresh

turns out the solution is to sync the workspace. make sure, the workspace you are synching with intended.

terraform workspace list

terraform workspace select master

Upgrade your chrome experience. Leverage on this app to handle tabs and urls navigation.

You can maximize your screen space, use Chrome in complete full screen mode by dropping off the address bar and the tab bar.

* type a URL followed by enter to visit the specified address
* type a term followed by enter to search the term (the search engine is configurable)
* type a term to navigate to any popular search suggestions
* type a keyword to navigate to any mapped URL (the map is configurable)
* type a term to navigate to any open tabs
* type a term to get the likely result you would like to visit


**Configurations**
change the search engine from the options page
configure the keyword and URL mapping

**Keyboard Shortcut:**
 
Windows: Alt+L

Mac: ⌥+L

https://chrome.google.com/webstore/detail/tab-url-manager/egiemoacchfofdhhlfhkdcacgaopncmi

git rebase -p –onto 6c31b73-commit-to-drop^  6c31b73-commit-to-drop

AWS S3 is a popular choice for cloud storage, due to its cost and stability.

It will be super convenient if we can mount the S3 bucket, so that to access it through FTP or from local OS directly.

Steps:

1. create the S3 bucket
2. set up the proper access
   1. either through s3 policy
   2. or through IAM policy
      1. if through IAM policy, need to create the role, associate the role with the policy
3. install s3fs either from source or package

   #ubuntu
   sudo apt install s3fs
   
   
   #from source
   

   git clone https://github.com/s3fs-fuse/s3fs-fuse.git
   cd s3fs-fuse
   ./autogen.sh
   ./configure
   make
   sudo make install

    

4. mount by role
   

    

   mkdir /mnt-drive && s3fs  -o iam_role=”role-from-step-2” -o allow_other S3-bucket /mnt-drive

    

5. mount by key

   1. mkdir /mnt-drive && s3fs -o passwd_file=${HOME}/.passwd-s3fs
      -o allow_other S3-bucket /mnt-drive

6. FTP

   1. install vsftpd

      systemctl start vsftpd

git filter-branch –tree-filter ‘rm -rf largeFile’ HEAD

ref: https://dalibornasevic.com/posts/2-permanently-remove-files-and-folders-from-a-git-repository

Full Screen your chrome, then leverage ont this plugin for URL box

Enjoying Chrome in Completely Full Screen mode even without omnibox (Zen). Leave the address typing to ZenHelper.

Keyboard Shortcut: 
windows: Ctrl+Shift+L
mac: Cmd+Shift+L


Note: This is still beta stage, new feature WIP.

https://chrome.google.com/webstore/detail/zenhelper/odpfhihammejhokpbbkbhlmhnoipglmc

https://chrome.google.com/webstore/detail/tab-url-manager/egiemoacchfofdhhlfhkdcacgaopncmi

https://dev.to/lyfolos/map-vs-list-comprehension-in-python-2ljj

Had exactly this discussion the other day among several strong opinioned programmer.

View at Medium.com

After all its technology both still in exist, and the exist is with a reason. It’s ultimately boil down to personal preference, but neither should be forced as superior.

https://www.higherperspectives.com/stop-arguing-with-people-2638754853.html

it’s not a completely novel requirement to get the status for asynchronous processing, for example, to get the file upload progress, to get the order status, alert for processing failures etc.

IMO, there are mostly two approaches for it,
1.synchronous
whether it’s through REST/GraphQL/Queue/DB/FS, if the client is polling for status, that’s a synchronous call, which kind of patched for the original fire and forget async process.

1. asynch
   this would still be a patch for the original async process, however, it’s really valid. since it’s a “new” requirement from the client to get the status. instead of polling, which almost removed the merits of the original async call, the client could expose a webhook for the server to post back status asynchronously. as such, even though it’s a patch, it will be patched async, leave both processes fire and forget.

https://stackoverflow.com/questions/54841672/get-status-of-asynchronous-invocationtype-event-aws-lambda-execution/58870421#58870421
https://docs.aws.amazon.com/step-functions/latest/dg/sample-project-job-poller.html

history is full of similarity.
this is such a repeat of WSDL2Java vs Java2WSDL debate. personally, it’s nothing black or white, right or wrong, similar to the ORM vs mybatis, it’s just opposite design suitable for individual use cases.

https://blog.mirumee.com/schema-first-graphql-the-road-less-travelled-cf0e50d5ccff

it might be true that Google or Facebook has some monorepo alive since more than a decade or two decades ago, however, that doesn’t means it’s the right approach for any new projects to adapt in 2019.

tech has been improving on loose coupling and SoC, which is a design idea since GOF and even earlier. this has continuously generating the benefits from version control, project segregation, building, CI/CD and micro services, which resulted faster and better quality delivery and long term maintenance. I will be surprised if monorepo would flourish in next decade or any near future.

quite interesting to see all those ideas/noises/debates come back and forth though.

View at Medium.com

it’s a dilemma, the reason why the language become more popular, (especially among the new or maybe amateur programmers) is due to its loose design, easy to learn (at the cost of not to leverage on the static typing, multi-threading concurrency, and JIT and AOTs for example), which is the exact reason for its being not as performing as the other languages.

https://hackernoon.com/why-is-python-so-slow-e5074b6fe55b

Its been around a decade since MS tried to emulate the success of java, ever since j++ to. Net c#.

Now

MS is a java shop now.

Java is not only de facto langue for thoughtful enterprise app, its ability to keep evolving from several years release cycle to half yearly, and with so many version poped up each with up to date features design principal, yet as the same time as backward compatible as possible (its unimaginable of the python versioning issue from a java world.) java is very mature stable language yet growing at even faster speed than any new lanagues with the very solid design carefully thought of language principle, its able to grow even faster and at the same time as always stable and trustworthy.

https://jaxenter.com/microsoft-ready-contribute-openjdk-163550.html

Years back, it was very much a hassle and careful thought process to build the right projet skeleton with the right code structure and inline libraries. After that it’s another equally if not more challenging to pave the runway for the poc to build test deploy and put on suitable scale of Web servers/ containers, application server, and databases with normalisation and vertical horizontal scaling.

All these efforts have now significantly saved thanks to all new frameworks which bundle all these together, providing an optioned suite.

With the popular of the frameworks and tools, then come with now a new challenges, which is to properly leverage on the tools.

Aws for example, has shoot up from single digit toolset size to now probably hundred.

Container has gone from vm chroot jail docker cluster k8s swam.

There is definitely value added to have the skills (DevOps or admin or infra) to pave the right platforms at this era.

However, I won’t be surprised as tide moves, there will be tools to bundle these tools together and save all these skills or effort in future. For example, the recent Pivotal conference to ask developers to forget k8s.
https://www.zdnet.com/article/devops-now-most-sought-after-skill-survey-finds/

https://www.zdnet.com/article/linus-torvalds-im-not-a-programmer-anymore/

https://thenewstack.io/pivotal-wants-to-let-developers-forget-about-kubernetes/

The default way to authen then talk with registry is through

docker login.

The user name is aws and password could be retrieve using

Aws ecr get-token

So far it’s pretty straightforward.

However, there is a caveat there. The token from aws CLI is valid for 12 hours only, this is aws’s approach to secure the access, in case the token is compromised, it’s to be expired then only authorised could retrieve the new token.

One possible approach to keep the docker CLI work is to refresh the

Docker login

Every 12 hours. Which is not difficult however is very ugly.

Instead, aws has this Credential helper. So with the Aws-ecr-Credential-helper installed, when we run docker CLI, it’s able to pick up the config from ~/.docker/config.json

"credHelpers": {
		"aws_account_id.dkr.ecr.region.amazonaws.com": "ecr-login"
	}

That it would leverage on the helper to talk to the specific ecr instance. And the helper in turn would leverage on pre-configured ~/.aws/credential & ~/.aws/config to pick up the right access key and secret etc to talk with ecr.

This is a cool solution not only for Docker CLI but actually a lot serverless platform as well which relies on containers.

https://aws.amazon.com/blogs/compute/authenticating-amazon-ecr-repositories-for-docker-cli-with-credential-helper/

Forbes: GraphQL And The Rise of Universal Data Access.
https://www.forbes.com/sites/cognitiveworld/2019/10/28/graphql-and-the-rise-of-universal-data-access/

Won’t be surprised there will be a autocode + automl someday.

https://www.forbes.com/sites/gregoryferenstein/2019/10/26/how-data-science-could-become-a-mainstream-career/

http://sandordargo.com/blog/2019/08/07/forty-and-still-a-dev

magalix.com: Kubernetes Patterns : The Stateful Service Pattern.
https://www.magalix.com/blog/kubernetes-patterns-the-stateful-service-pattern

https://medium.com/better-programming/programmers-make-yourself-replaceable-1b08a94bf5

https://dev.to/seattledataguy/top-signs-of-an-over-experienced-programmer-humor-satire-39hk

The delusion happens when developers start believing that declarative languages are better than imperative languages.

http://tutorials.jenkov.com/the-declarative-delusion.html

Personally, there is no perfect solution for all.
Nowadays, there are so many solutions/partial-solutions/broken-solutions, which for people, normally has a lazy tendacy, would like to fanatic easy to understand simply because it’s easy. But easy solution is normally suitable for special or specific domain, in some cases, its broken. Otherwise, for example, why the mostly widely used langue for enterprises is java?

https://smallbusinessprogramming.com/the-one-chart-every-developer-must-understand/

High-quality software is not expensive. High-quality software is faster and cheaper to build and maintain than low quality software, from initial development all the way through total cost of ownership