code-scan starred go-zero-looklook

go-zero-looklook

Summary

The old rule is to give a star before you see it. Ha ~ ~, if the star is caught, you have to spank, ha ha

I may have contacted go zero earlier. I have been using go zero since about 1000star. Later, I became familiar with the go zero author through wechat. The go zero author was very enthusiastic and patient to help me answer many questions. I also want to actively help go zero promote the community, Basically, I answered the relevant questions in the community group, because in this process, I found that many people felt that go zero did not have a complete project example. As a member of the community who wanted to promote the community, I made an available version open source. The main technology stack includes the following：

k8s
go-zero
nginx-gateway
filebeat
kafka
go-stash
elasticsearch
kibana
prometheus
grafana
jaeger
go-queue
asynq
asynqmon
dtm
docker
docker-compose
mysql
redis
gin-vue-admin
air
modd
jenkins
gitlab
harbor

Doc

https://github.com/Mikaelemmmm/go-zero-looklook/tree/main/doc Under the doc directory of the project

Docker compose is recommended in the development environment of the project. The direct chain method is used to give up the trouble caused by service registration and discovery middleware (etcd, nacos, consul ...)

For testing and online deployment, k8s (etcd, Nacos, consumer, etc.) has detailed tutorials (build + deploy), which can be communicated in the go zero community group, which is very easy

Project Summary

The whole project uses the micro services developed by go zero, which basically includes go zero and some middleware developed by relevant go zero authors. The technology stack used is basically the self-developed component of the go zero project team, which is basically the go zero software

The project directory structure is as follows：

admin：Background code (integrated with gin Vue admin, acting as a large background gateway) uses grpc to interact with RPC business under app. There is an example of code interaction between background gin Vue admin and go zero. If you don't want the background, you can directly delete the whole folder of admin and execute go mod tidy once
admin/web : Background web side code, gin-Vue-admin(https://github.com/flipped-aurora/gin-vue-admin)
app：All business codes include API, RPC and MQ (message queue, delay queue, scheduled task)
common：General components ( error、middleware、interceptor、tool、ctxdata ...)
data：The project contains the data generated by the directory relying on all middleware (mysql, ES, redis, grafana, ...). All contents in this directory should be in Git ignore files and do not need to be submitted.
deploy：
- filebeat: docker deployment filebeat config
- go-stash：go-stash config
- nginx: nginx-gateway config
- prometheus ： prometheus config
- script：
  - gencode：auto generate api、rpc，and use kafka script ，copy and paste use
  - mysql：auto generate model code shell script
- goctl: The template and goctl of the project generate custom code templates. For the usage of template, refer to the go zero document and copy it to the home directory Goctl is enough
doc : This project series documents
modd.conf : Don't be afraid of Modd hot loading configuration file. It's very simple to use. You can learn more about Modd usage here: https://github.com/cortesi/modd , the image of this project only uses golang-1.17.7-alpine as the basic image and installs Modd internally. If you want to add goctl, protoc, golint, etc., it's the same to directly create an image without my image

System architecture diagram

Business architecture diagram

Gateway

Nginx acts as a gateway and uses the auth module of nginx to call the identity service of the back-end for unified authentication. There is no authentication within the business. If there are a lot of business funds involved, you can also carry out secondary authentication in the business for security.

In addition, many students think nginx is not very good as a gateway. The principle is basically the same. They can replace it with apisik, Kong ..

Development mode

This project uses microservice development, API (HTTP) + RPC (grpc). The API acts as an aggregation service, and the complex and related business calls are uniformly written in RPC. If some simple businesses will not be relied on by other services, they can be directly written in the logic of API

Log

For logs, filebeat is used to collect them and report them to Kafka. Because logstash knows everything and the resource occupation is too exaggerated, go stash is used instead of logstash

Link: https://github.com/kevwan/go-stash , go stash is developed by the go zero development team. It has high performance and does not occupy resources. There is not much main code. It can be used only by configuration. It is very simple. The Kafka data source is synchronized to elasticsearch. The elasticsearch account and password are not supported by default. I fork a copy and modify it. It simply supports the account and password

Monitor

Prometheus is used for monitoring. This go zero native support only requires configuration. Here you can see the configuration in the project

Link tracking

Go zero supports Jaeger and Zipkin by default. You only need to configure it. You can see the configuration

Message queue

The message queue uses the go queue developed by the go zero development team. The link is: https://github.com/zeromicro/go-queue

KQ can be used here. KQ is a high-performance message queue based on Kafka

DQ queue is used in the current project, but there is no delay in DQ queue

Delay queue, Scheduled task

Delay queue and scheduled tasks this project uses asynq, a simple middleware developed based on redis,

Of course, asynq also supports message queues. You can also replace KQ message queues with this one. After all, it's good to only need redis without maintaining a Kafka

Link：https://github.com/hibiken/asynq

Distributed transaction

DTM is used for distributed transactions. Well, it's very comfortable. I wrote a "go zero nanny tutorial on connecting distributed transactions with DTM" link address: https://github.com/Mikaelemmmm/gozerodtm , this project has not been used yet. It's good to prepare for direct integration in the future. If readers use it, they can directly look at the source code

Dtm Link : https://github.com/dtm-labs/dtm

Deploy

For testing and online deployment, k8s (etcd, Nacos, consumer, etc.) has detailed tutorials (build + deploy), which can be communicated in the go zero community group, which is very easy

project doc ：https://github.com/Mikaelemmmm/go-zero-looklook/tree/main/doc

gitlab + jenkins + harbor + k8s

Click deploy the corresponding service in Jenkins, and you will go to gitlab to pull the code -- > and then pull the online configuration (a separate git library is configured online, why not use the configuration center, which is described in the deployment document) - -- > automatically build the image -- > push it to the harbor image warehouse -- > automatically publish it to k8s using kubectl -- > a nignx should be hung in front as the gateway for unified entry, authentication, current restriction ....

Thanks

go-zero : https://github.com/zeromicro/go-zero

dtm：https://github.com/dtm-labs/dtm

gin-vue-admin：https://github.com/flipped-aurora/gin-vue-admin