You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 9 Next »

Steps to upgrade DSS Cloud (with Kubernetes and User Isolation) to latest version.

For commands with sudo, run as your user (i.e. run as cheryl_abundo) instead of as dataiku.

For all other commands, run as dataiku.


  1. Check if anyone is logged in to DSS cloud 

    If no one is connected, stop DSS (run commands as dataiku)
    sudo su - dataiku
       ./design/bin/dss stop
        
    If someone is connected, inform the person/s before stopping DSS,

  2. Clean-up DSS disk
      rm -rf /home/dataiku/design/tmp/*
      rm -rf  /home/dataiku/design/caches/*
      rm -rf  /home/dataiku/design/exports/*
      rm -rf  /home/dataiku/design/diagnosis/*
      rm -rf  /home/dataiku/design/jobs/*
      rm -rf  /home/dataiku/design/scenarios/*

  3. Backup data directory

    1. Check the size of the DSS data directory
      du -hcs  /home/dataiku/design


    2. Check if there's enough disk space in the VM
      df -h

      If there's not enough space, increase disk by following 3c else proceed to 4

    3. Increase disk attached to VM instance


      1. Open boot disk and click on Edit. Specify higher Size and click Save.


      2. Open VM SSH and identify the disk with the file system and the partition that you want to resize.
        sudo lsblk

      3. Resize the image partition identified above.
        sudo growpart /dev/sda 1

      4. Extend the file system on the disk/partition to use the added space.
        sudo xfs_growfs /dev/sda1

      5. Verify that the file system is resized
        df -h /dev/sda1

    4. Copy DSS data directory to backup directory (can take awhile, have coffee, work on something else in the meantime)
        cp -rv ./design ./design_backup
      Check that the backup directory is about the same size  (=>) as the original data directory
        du -hcs design_backup/
        du -hcs design/
  4. Download the latest Dataiku installation file
      wget https://cdn.downloads.dataiku.com/public/dss/6.0.1/dataiku-dss-6.0.1.tar.gz
    Unpack the file
      tar xzf dataiku-dss-6.0.1.tar.gz
    After successfully unpacking, delete the tar file
      rm dataiku-dss-6.0.1.tar.gz

  5. Perform the upgrade
      dataiku-dss-6.0.1/installer.sh -d design -u

    If there is a missing dependency, run the following as your user (ie. cheryl_abundo)
      sudo -i "/home/dataiku/dataiku-dss-6.0.1/scripts/install/install-deps.sh" -without-java -with-conda
      Then as dataiku, rerun
    dataiku-dss-6.0.1/installer.sh -d design -u

    Successful update installation will show


  6. Update R installation
      ./design/bin/dssadmin install-R-integration


  7. Reinstall graphics export
      ./design/bin/dssadmin install-graphics-export
    If there is a missing dependency, run the following as your user (ie. cheryl_abundo)
      sudo -i "/home/dataiku/dataiku-dss-6.0.1/scripts/install/install-deps.sh" -without-java -without-python -with-chrome
      Then as dataiku, rerun
    ./design/bin/dssadmin install-graphics-export

  8. Reinstall standalone Hadoop and Spark

    1. Download required files
      wget https://cdn.downloads.dataiku.com/public/dss/6.0.2/dataiku-dss-spark-standalone-6.0.2-2.4.3-generic-hadoop3.tar.gz
      wget https://cdn.downloads.dataiku.com/public/dss/6.0.2/dataiku-dss-hadoop3-standalone-libs-generic-6.0.2.tar.gz

    2. Run Hadoop integration
      ./design/bin/dssadmin install-hadoop-integration -standalone generic-hadoop3 -standaloneArchive /home/dataiku/dataiku-dss-hadoop3-standalone-libs-generic-6.0.2.tar.gz

    3. Run Spark integration
      ./design/bin/dssadmin install-spark-integration -standaloneArchive /home/dataiku/dataiku-dss-spark-standalone-6.0.2-2.4.3-generic-hadoop3.tar.gz

    4. Build container images (can take awhile, have coffee or work on something else in the meantime)
      ./design/bin/dssadmin build-container-exec-base-spark-image

    5. Build code env images
      ./design/bin/dssadmin build-code-env-images --all

    6. Build base images
              ./design/bin/dssadmin build-mad-kubernetes-base-image

  9. Secure the new installation (user isolation framework)
    Run as your user (ie. cheryl_abundo)
      sudo /home/dataiku/design/bin/dssadmin install-impersonation dataiku 

  10. Start DSS
      ./design/bin/dss start

  11. Configure container execution

    Click on PUSH IMAGES after

  12. Configure Spark

 

  

  Config keys to add:

  spark.master → k8s://https://35.241.222.100:443
  spark.kubernetes.container.image → gcr.io/bi-gcp-sandbox/dku-spark-base-tio1uka1wnxmwgtnxslnjrxl:dss-6.0.2

  

  Click on PUSH IMAGES after


 

 

  • No labels