如何使用Jenkins搭建基本的发布系统

在搭建自主发布系统之前,我们已经默认你已经使用代码托管仓库和包管理中心。
自主搭建Gitlab作为代码仓库;
自主搭建Maven作为Jar包管理中心;

 

发布系统的主要功能分为:
  1. 基础服务的编译
  2. 基础服务的发布
  3. 服务的回滚

一、服务编译:

思路:是用户填写一个Git代码的分支号和CommitId,系统从Git仓库中拉去相应的分支代码,并进行编译打包。

第一步:创建Job

在Jenkins中新建一个自由风格的软件项目,使用参数化构建,让用户可以填写分支号和CommitID:

jenkins-job-build-param
构建Shell(将用户填写的基本参数信息传递给本地shell脚本):
#echo $JOB_NAME $DeployEnv $BUILD_USER
build_name=${BUILD_NUMBER}-$JOB_NAME-$BUILD_USER_ID
cd /home/admin/.jenkins/git_repository
sh onlyBuild.sh "$JOB_NAME" "$branchName" "$commitID" "$buildType" "$BUILD_URL" "$build_name"

当然,构建后操作中可以添加邮件提醒,这里就不再配置。

第二步:执行构建

Job创建完毕后,是这样的:
jenkins-service-build-interface
onlyBuild.sh脚本:
#!/bin/sh
###############################################
##  Only pull code and generate the package
##  Read paraments
###############################################
#LOGFILE="/tmp/onlyBuild.log"
#:>>"$LOGFILE"
#exec 1>"$LOGFILE"
#exec 2>&1
source /etc/profile
projectName=$1
branchName=$2
version=$3
buildType=$4
build_url=$5
build_name=$6
gruntVersion=$7
deploy=$8
serviceName=`echo $projectName|awk -F"-build" '{print $1}'`
base_dir=/home/admin/.jenkins/git_repository
buildRecordDir=/home/admin/build_history/$serviceName

################################################
##   get the times of build
################################################
buildDate=`date +%m%d-%H:%M`
cd $base_dir
sh -x buildGitManagere.sh $serviceName $branchName $version
if [ $? -eq 0 ]; then
   echo "SUCCESS: Update $serviceName code successfully"
   echo "SUCCESS: 更新代码成功"
else
   echo "ERROR: Update $serviceName code fautly"
   echo "ERROR: 更新代码失败"
   exit -100
fi

################################################
##  package
################################################
cd $base_dir
if [ x"$buildType" = x"grunt" ]; then
   cd $base_dir
   sh -x deployWEBH5.sh $serviceName
   if [ $? -eq 0 ]; then
       echo "SUCCESS: $serviceName 打包成功."
   else
       echo "ERROR: $serviceName 打包失败."
       exit -100
   fi
elif [ x"$buildType" = x"mvn" ]; then
   cd $base_dir/$serviceName
   grep "$serviceName" $base_dir/autoconfig_map
   if [ $? -eq 0 ]; then
      mvn clean install -DskipTests -Dautoconfig.skip
      if [ $? -eq 0 ]; then
         echo "SUCCESS: mvn $serviceName package success"
         echo "SUCCESS: 服务打包编译成功."
         echo "ServiceName: $serviceName" > lastBuildNote
         echo "BuildDate: $buildDate" >> lastBuildNote
         echo "BranchName: $branchName" >> lastBuildNote
         echo "Version: $version" >> lastBuildNote
         echo "build_name: $build_name" >> lastBuildNote
         echo "build_url: $build_url" >> lastBuildNote
      else
         echo "ERROR: mvn $serviceName package fautly, please check mvn log."
         echo "ERROR: 服务打包编译失败."
         exit -100
      fi

   else
      #get config properties
      ####################################
      # assert the service
      ####################################
      conf_dir=$base_dir/service.conf/$serviceName
      conf_file=$conf_dir/$serviceName.conf
      #cd $conf_dir
      if [ -d "$conf_dir" ] && [ -e "$conf_file" ]; then
         echo "[SUCCESS]: Find $serviceName config file at $conf_dir, reading config file...."
      else
         echo "[ERROR]: 不能成功读取该服务的相关配置文件";
         exit -100
      fi

      cd $base_dir
      #deployEnv=`grep deployENV $base_dir/host_map |awk -F: '{print $NF}'`
      deployEnv=`grep ENV /home/admin/bin/config.sh|awk -F "=" '{print $NF}'`
      grep configFile $conf_file
      if [ $? -eq 0 ]; then
         confName=`grep configFile $conf_file|awk -F"=" '{print $2}'`
         conf_file_name=`ls $confName|awk -F"/" '{print $NF}'`
         echo "[SUCCESS]: $serviceName's config file is $conf_file_name"
         cd $base_dir && sh -x replaceConfig.sh $serviceName $deployEnv $confName
         if [ $? -eq 0 ]; then
            echo "SUCCESS: 替换配置文件成功."
         else
            echo "ERROR: 替换配置文件失败."
            exit -100
         fi
      else
         echo "ERROR: 没有发现$serviceName."
         exit -100
      fi

      cd $base_dir/$serviceName
      /opt/haitao/install/apache-maven-3.2.3/bin/mvn clean install -DskipTests
      if [ $? -eq 0 ]; then
         echo "SUCCESS: 服务打包编译成功."
      else
         echo "ERROR: 服务打包编译失败."
         exit -100
      fi
   fi
else
   echo "ERROR: 未知的项目类型"
   exit -100
fi

#copy package to version_repository
cd $base_dir
sh -x mvpackage.sh $serviceName
if [ $? -ne 0 ]; then
   echo "ERROR: 克隆包失败."
   exit -100
fi

if [ x$deploy = xfalse ]; then
   echo "WARN: 暂停部署服务."
else
   #启动服务
        echo "启动$serviceName"
   sh -x callDeploy.sh $serviceName
   if [ $? -ne 0 ]; then
      echo "ERROR: 部署服务$serviceName失败."
      exit -100
   fi
fi
主要流程为:1.从Git仓库中拉去代码;2. AutoConfig文件替换;3. 编译代码;
buildGitManagere.sh 脚本从Git仓库中拉去代码,并主动跟Master做代码合并:
#!/bin/sh
service_name=$1
branchName=$2
version=$3
baseDir=/home/admin/.jenkins/git_repository

git config --global user.email “xxx@xxx.com"
git config --global user.name “xxx"
#cd service directory
service_dir=$baseDir/$service_name
if [ ! -n "$version" ]; then
   #version=null
   echo "ERROR: 没有指定对应的版本号,请重新指定."
   exit -100
fi
if [ -d $service_dir ]; then
    cd $service_dir
else
    if [ $? -ne 0 ]; then
       echo "WARN: direction $baseDir/$service_name is not exist."
       echo "WARN: 开始从gitlab clone项目$service_name."
       cd $baseDir && git clone git@gitlab.xxxxx.com:xxxx/$service_name.git
       if [ $? -ne 0 ]; then
          echo "ERROR: 不能从gitlab上clone项目$service_name."
          exit -100
       else
          cd $service_dir
       fi
   fi
fi

#assert remote branch
branch_name=$branchName
err_msg=`git reset --hard && git checkout master 2>&1 && git pull 2>&1`
if [ $? -ne 0 ]; then
   echo "ERROR: git checkout master and pull code error"
   echo "ERROR: $err_msg"
   exit -100
fi

#git branch -a|grep remotes|awk -F"/" '{print $NF}'|grep "^$branch_name$" 2>&1
git branch -a|grep "remotes/origin/$branch_name" >/dev/null 2>&1
if [ $? -ne 0 ]; then
    echo "ERROR: git origin remote don't have this branch"
    echo "ERROR: 远端主机没有$branch_name分支"
    exit -100
fi

if [ x"$branch_name"  = x"master" ]; then
     branch_version=`git log|sed -n '1p'|awk '{print $2}'`
     if [ x${branch_version} = x${version} ]; then
         echo "SUCCESS: Repository $service_name branch $branchName lasted version is $version"
     else
         echo "ERROR: Repository $service_name branch $branchName lasted version is  $branch_version not $version"
         echo "ERROR: 远程主机分支$branchName的版本和发布版本不一致"
         exit -100
     fi
else
    git branch|grep $branch_name
    if [ $? -eq 0 ]; then
       git branch -D $branch_name
       if [ $? -ne 0 ]; then
          echo "ERROR: 删除分支$branch_name失败."
          exit -100
       else
          echo "SUCCESS: 删除分支$branch_name成功."
       fi
    fi

    git checkout $branch_name && git reset --hard
    if [ $? -ne 0 ]; then
        echo "ERROR: 切换分支$branch_name失败."
        exit -1
    else
        echo "SUCCESS: 切换分支$branch_name成功."
    fi

    git pull origin $branch_name:$branch_name
    #err_msg=`git pull origin $branch_name:$branch_name 2>&1`
    if [ $? -ne 0 ]; then
        echo "ERROR: Repository $service_name $branch_name pull code error"
        echo "ERROR: 远程主机分支$branch_name拉取代码失败"
        #echo "ERROR: $err_msg"
        exit -1
    else
        echo "SUCCESS: Repository $service_name $branch_name pull code success"
    fi

    err_msg=`git merge master 2>&1`
    if [ $? -ne 0 ]; then
       echo "ERROR: Repository $Repository $branch_name merge master error."
       echo "ERROR: 远程主机分支$branch_name Merge 主分支失败."
       exit -100
    else
       echo "SUCCESS: Repository $Repository $branch_name merge master success."
    fi

    branch_version=`git log|sed -n '1p'|awk '{print $2}'`
    if [ x${branch_version} = x${version} ]; then
        echo "SUCCESS: Repository $service_name branch $branchName lasted version is $version"
    else
        echo "ERROR: Repository $service_name branch $branchName lasted version is  $branch_version not $version"
        echo "ERROR: 远程分支 $branchName 最新的代码是 $branch_version 不是 $version"
        exit -100
    fi

fi
replaceConfig.sh 脚本就是从Diamond上拉去AutoConfig配置文件替换本地的文件变量。
#!/bin/sh
base_dir=/home/admin/.jenkins/git_repository
servicerName=$1
env=$2
confName=$3
diamondServer=`grep autoconfig-diamond $base_dir/host_map |awk -F: '{print $NF}'`
diamondPort=8888

#check service is exist
cd $base_dir
mkdir -p $base_dir/config
if [ $? -eq 0 ]; then
   echo "SUCCESS: mkdir config directory success."
else
   echo "ERROR: mdkir config directory fautly."
   exit -100
fi

#get config file from diamond
wget_url="http://$diamondServer:$diamondPort/diamond-server/config.co?group=$env&dataId=$servicerName"
inconv_file=$servicerName.config.properties
target_config_dir=$base_dir/$confName
wget $wget_url -T 5 -t 3 -O ./config/$servicerName.config.properties.bak
if [ $? -eq 0 ]; then
   echo "wget $servicerName's config file from diamond success"
else
   echo "wget $servicerName's config file from diamond failed"
   exit -1
fi

#iconv gbk file to utf-8 file
iconv -f gbk -t utf-8 -o ./config/$servicerName.config.properties ./config/$servicerName.config.properties.bak
if [ $? -eq 0 ]; then
   echo "Convert $servicerName's config file success"
else
   echo "Convert $servicerName's config file failed"
   exit -1
fi

#cp file to target dir
cp ./config/$servicerName.config.properties $target_config_dir
if [ $? -eq 0 ]; then
   echo "Replace $servicerName's config file success"
   echo "finish to replace $servicerName's config file"
else
   echo "Replace $servicerName's config file failed"
   exit -1
fi

注意:

  在生产环境,由于合并到master上的代码中的配置文件必须是没有经过配置文件替换的,所以一般采用的流程是:编译过程只做拉代码和编译,预发环境拿编译出来的包进行配置文件替换(预发环境不需要合并主干),再部署到预发环境,生产环境发布的时候需要拿编译出来的包先进行合并Master主干,Push到远程,再进行配置文件替换和部署操作。

二、服务发布:

第一步:创建Job
一般的Maven工程会有多个module,我们可能会选择其中的一个进行发布,如果线上有多台机器,我可能会希望每次只发布一台,所以我们可能需要的页面是这样子的:
jenkins-deploy-service-deploy-preview
可以选择我们期望发布的环境,选择我们期望发布的机器等。
那怎么获取服务的module信息呢?怎么获取服务对应的机器列表信息呢?
使用Dynamic Choice Parameter 配置模块下拉(使用Groovy脚本):
jenkins-deploy-package-moudle
def build = Thread.currentThread().toString()
def regexp= ".+?/job/([^/]+)-deploy/.*"
def match = build  =~ regexp
def jobName = match[0][1]
def StringBuilder serverNameBuilder= new StringBuilder("");
 serverNameBuilder.append(" cd /home/admin/.jenkins/git_repository/");
 serverNameBuilder.append(jobName);
 serverNameBuilder.append("; ls -l|grep  '^d'|awk  '{print $9}'");
def String fullServerName = serverNameBuilder.toString();
def proca = [ 'bash', '-c',fullServerName ]
proca.execute().text.tokenize('n')

 

 配置单选环境选择下拉框:
 jenkins-deploy-env-params
配置可多选的服务器IP地址列表:
jenkins-deploy-hostlist
def String service = binding.variables.get("packageMoudle")
def String deployEnv= binding.variables.get("deployEnv")

def StringBuilder serverNameBuilder= new StringBuilder("");
 serverNameBuilder.append("sh /home/admin/getPrdIp/getIp.sh ");
 serverNameBuilder.append(service );
 serverNameBuilder.append(" "+deployEnv);
def String fullServerName = serverNameBuilder.toString();
def proca = [ 'bash', '-c',fullServerName ]
hosts = proca.execute().text.replaceAll(/^s*/, '').replaceAll(/s*$/, '').split(' ').toList()
return hosts

 

这里引用的参数是上面选择的模块和环境,不同的模块和环境对应不同的机器IP列表。这里面的原理还算比较简单,使用 binding.variables.get(“”)方法获取自定义变量,通过StringBuilder拼接脚本命令,最后用[ ‘bash’, ‘-c’,fullServerName ] 定义bash脚本,并执行bash脚本获取结果后对结果进行替换或分割。

构建Shell脚本:

build_name=${BUILD_NUMBER}-$JOB_NAME-$BUILD_USER_ID
cd /home/admin/.jenkins/git_repository
sh onlyDeploy.sh "$JOB_NAME" "$packageMoudle" "$commitID" "$deployEnv" "$hostlist" "$BUILD_URL" "$build_name" "$releaseNote"

第二步:服务部署

部署服务的界面上面已经描述,那么这里将要具体描述一下部署脚本的写法。

三、服务回滚:

思路:每次部署部署的时候都把发布的包拷贝一份到备份目录中,下一次可以直接选择这个目录中的包进行部署操作。
那么我们期望的回滚操作页面可能是这样子的:
 jenkins-rollback-interface
有环境列表,有服务模块,有发布的Hashcode,还有机器列表。
首选需要创建一个这样的Job,上面已经描述了怎么配置环境列表、服务模块、机器列表,这里主要将怎么获取发布的commitID:
jenkins-service-rollback-commitids
def String jobName= binding.variables.get("serverName");
def String deployEnv = binding.variables.get("deployEnv");
def String packageMoudle = binding.variables.get("packageMoudle");
//def
def StringBuilder serverNameBuilder= new StringBuilder("");
 serverNameBuilder.append(" cd /home/admin/backup/");
 serverNameBuilder.append(deployEnv);
 serverNameBuilder.append("/");
 serverNameBuilder.append(jobName);
 serverNameBuilder.append("/");
 serverNameBuilder.append(packageMoudle);
 serverNameBuilder.append("; ls -lt|grep  '^d'|awk  '{print $9}'");
def String fullServerName = serverNameBuilder.toString();
def proca = [ 'bash', '-c',fullServerName ]
proca.execute().text.tokenize('n')

 

Advertisements

发表评论

Fill in your details below or click an icon to log in:

WordPress.com 徽标

You are commenting using your WordPress.com account. Log Out /  更改 )

Google photo

You are commenting using your Google account. Log Out /  更改 )

Twitter picture

You are commenting using your Twitter account. Log Out /  更改 )

Facebook photo

You are commenting using your Facebook account. Log Out /  更改 )

Connecting to %s