[MSA] Spring Cloud로 MSA를 개발해보자 15편 [배포]

1. 서비스 실행 방법

- IntelliJ IDEA

- Exported JAR file

- Docker Container

 

2. Docker Network

1). Bridge network

$ docker network create --driver bridge [브릿지이름]

2). Host network

  - 네트워크를 호스트로 설정하면 호스트의 네트워크 환경을 그대로 사용

  - 포트 포워딩 없이 내부 어플리케이션 사용

3). None network

  - 네트워크를 사용하지 않음

  - Io 네트워크만 사용, 외부와 단절

 

4). docker 기본 네트워크 목록

$ docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
b0d1b31fd518   bridge    bridge    local
c4feea674aa2   host      host      local
a4f9aa079695   none      null      local

 

5). bridge network 생성

$ docker network create --gateway 172.18.0.1 --subnet 172.18.0.0/16 ecommerce-network
$ docker network ls
// network 상세 정보
$ docker network inspect ecommerce-network

 

3. 서비스 배포

1). 공통사항

pom.xml 버전을 1.0으로 모두 변경하기

2). RabbitMQ

$ docker run -d --name rabbitmq --network ecommerce-network -p 15672:15672 -p 5672:5672 -p 15671:15671 -p 5671:5671 -p 4369:4369 -e RABBITMQ_DEFAULT_USER=guest -e RABBITMQ_DEFAULT_PASS=guest rabbitmq:management

 

127.0.0.1로 RabbitMQ에 접속할 수 있는 이유는 host pc에서 15672 port를 open하게되면

rabbitmq:15672에 연결되게 포트포워딩을 해놓은 상태라서 그렇다

 

3). Configuration Service

key 파일을 프로젝트에 넣기

(1). Dockerfile

FROM openjdk:17-ea-jdk-slim
VOLUME /tmp
COPY apiEncryptionKey.jks apiEncryptionKey.jks
COPY target/config-service-1.0.jar ConfigServer.jar
ENTRYPOINT ["java", "-jar", "ConfigServer.jar"]

 

 

rabbitMQ host ip를 application.yml에서 변경해도 되지만 run할때 rabbitmq의 name을 적으면 된다

 

(2). bootstrap.yml

encrypt:
  key-store:
    location: file:/apiEncryptionKey.jks // 경로 변경
    password: test1234
    alias: apiEncryptionKey

 

(3). application.yml

  cloud:
    config:
      server:
        native:
          search-locations: file:///C:/Users/PC/Desktop/project/native-file-repo
        git:
          uri: https://github.com/GoonManDoo/spring-cloud-config
          default-label: master

 

$ mvn clean compile package -DskipTests=true

$ docker build -t noory/config-service:1.0 .
$ docker images
REPOSITORY             TAG          IMAGE ID       CREATED          SIZE
noory/config-service   1.0          7eb252b1a642   14 seconds ago   446MB

$ docker run -d -p 8888:8888 --network ecommerce-network -e "spring.rabbitmq.host=rabbitmq" -e "spring.profiles.active=native" --name config-service noory/config-service:1.0
$ docker ps -a

 

4). Eureka Discovery Service

(1). Dockerfile

FROM openjdk:17-ea-jdk-slim
VOLUME /tmp
COPY target/discoveryservice-1.0.jar DiscoveryService.jar
ENTRYPOINT ["java", "-jar", "DiscoveryService.jar"]

 

$ mvn clean compile package -DskipTests=true
$ docker build --tag noory/discovery-service:1.0 .
$ docker images
$ docker push noory/discovery-service:1.0
$ docker push noory/config-service:1.0

$ docker run -d -p 8761:8761 --network ecommerce-network -e "spring.cloud.config.uri=http://config-service:8888" --name discovery-service noory/discovery-service:1.0

 

 

5). ApiGateway Service

(1). Dockerfile

FROM openjdk:17-ea-jdk-slim
VOLUME /tmp
COPY target/apigateway-service-1.0.jar ApigatewayService.jar
ENTRYPOINT ["java", "-jar", "ApigatewayService.jar"]

 

$ mvn clean compile package -DskipTests=true
$ docker build --tag noory/apigateway-service:1.0 .
$ docker images
$ docker push noory/apigateway-service:1.0

$ docker run -d -p 8000:8000 --network ecommerce-network -e "spring.cloud.config.uri=http://config-service:8888" -e "spring.rabbitmq.host=rabbitmq" -e "eureka.client.serviceUrl.defaultZone=http://discovery-service:8761/eureka/" --name apigateway-service noory/apigateway-service:1.0

 

6). MariaDB

(1). Dockerfile

// C:\Program Files\MariaDB 10.5\data를 /mysql_data 폴더를 만든 뒤 복사해준다

FROM mariadb:10.5.26
ENV MYSQL_ROOT_PASSWORD=root
ENV MYSQL_DATABASE=msa_kafka
COPY ./mysql_data/data /var/lib/mysql
EXPOSE 3307
ENTRYPOINT ["mysqld", "--user=root"]

 

// 지우고 다시 빌드하는 명령어
$ docker stop mariadb
$ docker rm mariadb
$ docker rmi 이미지ID


// 빌드부터 시작하는 명령어
$ docker build -t noory/my_mariadb:1.0 .
$ docker images
$ docker push noory/my_mariadb:1.0

$ docker run -d -p 3307:3307 --network ecommerce-network --name mariadb noory/my_mariadb:1.0

// MariaDB 접속 | powershell에서 하기
$ docker exec -it mariadb /bin/bash
# mysql -h127.0.0.1 -uroot -p

 

7). Kafka

Zookeeper + Kafka를 docker-compose로 한번에 실행할거다

 

(1). git clone : https://github.com/wurstmeister/kafka-docker.git

GitHub - wurstmeister/kafka-docker: Dockerfile for Apache Kafka

$ git clone https://github.com/wurstmeister/kafka-docker.git

 

(2). docker-compose-single-broker.yml

version: '2'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - "2181:2181"
    networks:
      my-network:
        ipv4_address: 172.18.0.100
  kafka:
    # build: .
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: 172.18.0.101
      KAFKA_CREATE_TOPICS: "test:1:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    // zookeeper이 실행되고 kafka가 실행되야 되기때문에 중요한 설정
    depends_on:
      - zookeeper
    networks:
      my-network:
        ipv4_address: 172.18.0.101

networks:
  my-network:
    external: true
    name: ecommerce-network

 

// docker-compose 경로에서
// 실행
PS C:\Work\docker-files\kafka-docker> docker-compose -f docker-compose-single-broker.yml up -d

// 종료
PS C:\Work\docker-files\kafka-docker> docker-compose -f docker-compose-single-broker.yml down -d

 

8). Zipkin

$ docker run -d -p 9411:9411 --network ecommerce-network --name zipkin openzipkin/zipkin

 

9). 모니터링

(1). Prometheus.yml

// target명을 localhost -> docker name으로 변경
    static_configs:
      - targets: ["prometheus:9090"]
  - job_name: 'user-service'
    scrape_interval: 15s
    metrics_path: '/user-service/actuator/prometheus'
    static_configs:
    - targets: ["apigateway-service:8000"]
  - job_name: 'apigateway-service'
    scrape_interval: 15s
    metrics_path: '/actuator/prometheus'
    static_configs:
    - targets: ["apigateway-service:8000"]
  - job_name: 'order-service'
    scrape_interval: 15s
    metrics_path: '/order-service/actuator/prometheus'
    static_configs:
    - targets: ["apigateway-service:8000"]

// Prometheus
$ docker run -d -p 9090:9090 --network ecommerce-network --name prometheus -v /c/Work/prometheus-2.53.2.windows-amd64/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
 
// Grafana
$ docker run -d -p 3000:3000 --network ecommerce-network --name grafana grafana/grafana

 

10). User-Service

(1). User-service.yml

// docker gateway ip
gateway:
  ip: 172.18.0.5

 

(2). Dockerfile

FROM openjdk:17-ea-jdk-slim
VOLUME /tmp
COPY target/user-service-1.0.jar UserService.jar
ENTRYPOINT ["java", "-jar", "UserService.jar"]

$ mvn clean compile package -DskipTests=true
$ docker build -t noory/user-service:1.0 .
$ docker run -d --network ecommerce-network --name user-service -e "spring.cloud.config.uri=http://config-service:8888" -e "spring.rabbitmq.host=rabbitmq" -e "spring.zipkin.base-url=http://zipkin:9411" -e "eureka.client.serviceUrl.defaultZone=http://discovery-service:8761/eureka/" -e "logging.file=/api-logs/users-ws.log" noory/user-service:1.0

 

11). Order-Service

(1). KafkaProducerConfig.java

    @Bean
    public ProducerFactory<String, String> producerFactory() {
        Map<String, Object> properties = new HashMap<>();
        // 로컬
//        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
        // docker kafka
        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "172.18.0.101:9092");
        properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);

        return new DefaultKafkaProducerFactory<>(properties);
    }

 

$ mvn clean compile package -DskipTests=true
$ docker build -t noory/order-service:1.0 .
$ docker run -d --network ecommerce-network --name order-service -e "spring.zipkin.base-url=http://zipkin:9411" -e "eureka.client.serviceUrl.defaultZone=http://discovery-service:8761/eureka/" -e "spring.datasource.url=jdbc:mariadb://mariadb:3306/msa_kafka" -e "logging.file=/api-logs/orders-ws.log" noory/order-service:1.0

// DB 접속 오류
PS C:\Work\docker-files\kafka-docker> docker exec -it mariadb /bin/bash
root@8bf7e910798e:/# mysql -uroot -p
Enter password:

MariaDB [(none)]> use mysql;
MariaDB [(none)]> grant all privileges on *.* to 'root'@'%' identified by 'root';
MariaDB [(none)]> flush privileges;

 

12). Catalog-Service

(1). KafkaConsumerConfig.java

    @Bean
    public ConsumerFactory<String, String> consumerFactory() {
        Map<String, Object> properties = new HashMap<>();
        // 로컬
//        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
        // docker kafka ip
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "170.18.0.101:9092");
        properties.put(ConsumerConfig.GROUP_ID_CONFIG, "consumerGroupId");
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);

        return new DefaultKafkaConsumerFactory<>(properties);
    }

 

(2). Dockerfile

FROM openjdk:17-ea-jdk-slim
VOLUME /tmp
COPY target/catalog-service-1.0.jar CatalogService.jar
ENTRYPOINT ["java", "-jar", "CatalogService.jar"]

$ mvn clean compile package -DskipTests=true
$ docker build -t noory/catalog-service:1.0 .

$ docker run -d --network ecommerce-network --name catalog-service -e "eureka.client.serviceUrl.defaultZone=http://discovery-service:8761/eureka/" -e "logging.file=/api-logs/catalogs-ws.log" noory/catalog-service:1.0

 

 

4. 서비스별 IP 및 포트 (순서대로 기동)

Service(Container name)	IP	Port
RabbitMQ	172.18.0.2/16	5671
config-service	172.18.0.3/16	8888
discovery-service	172.18.0.4/16	8761
apigateway-service	172.18.0.5/16	8000
mariaDB	172.18.0.6/16	3306
zookeeper	172.18.0.100/16	2181
kafka	172.18.0.101/16	9092
zipkin	172.18.0.7/16	9411
prometheus	172.18.0.8/16	9090
grafana	172.18.0.9/16	3000
user-service	172.18.0.10/16	RandomPort
order-service	172.18.0.11/16	RandomPort
catalog-service	172.18.0.12/16	RandomPort