轻松搞定,数据开发与分析也能如此简单方便!
一、引言
DataWorks是一站式智能大数据开发治理平台,深度适配阿里云MaxCompute、EMR、Hologres、Flink、PAI 等数十种大数据和AI计算服务,为数据仓库、数据湖、OpenLake湖仓一体、Data+AI解决方案提供全链路智能化的数据集成、大数据AI一体化开发、数据分析与主动式数据资产治理服务,帮助企业进行Data+AI全生命周期数据管理。
二、操作教程
产品开通
创建个人开发环境
应用体验
2.点击SQL Cell的请选择计算资源,在弹出框下选择要绑定的计算资源,若您在MaxCompute下无资源,则需点击“+绑定计算资源”
SELECT 'James' AS name, '25' AS age, 'Hangzhou' AS city;SELECT 'James' AS name, '25' AS age, 'Hangzhou' AS city;SELECT 'James' AS name, '25' AS age, 'Hangzhou' AS city;SELECT 'James' AS name, '25' AS age, 'Hangzhou' AS city;-- @conf name = flink_vvp_job_quick_start-- @conf engineVersion=vvr-8.0.8-flink-1.17-- @conf flinkConf."execution.checkpointing.interval"=10second-- @conf flinkConf."taskmanager.numberOfTaskSlots"=4-- @conf flinkConf."table.exec.state.ttl"=1hour-- @conf flinkConf."execution.checkpointing.min-pause"=10second-- @conf streamingResourceSetting.resourceSettingMode=BASIC-- @conf streamingResourceSetting.basicResourceSetting.parallelism=4-- @conf streamingResourceSetting.basicResourceSetting.taskmanagerResourceSettingSpec.memory=4GiB-- @conf streamingResourceSetting.basicResourceSetting.taskmanagerResourceSettingSpec.cpu=1-- @conf streamingResourceSetting.basicResourceSetting.jobmanagerResourceSettingSpec.memory=4GiB-- @conf streamingResourceSetting.basicResourceSetting.jobmanagerResourceSettingSpec.cpu=1--创建临时源表datagen_source。CREATE TEMPORARY TABLE datagen_source(randstr VARCHAR) WITH ('connector' = 'datagen' -- datagen连接器);--创建临时结果表print_table。CREATE TEMPORARY TABLE print_table(randstr VARCHAR) WITH ('connector' = 'print', -- print连接器'logger' = 'true' -- 控制台显示计算结果);--将randstr字段截取后打印出来。INSERT INTO print_tableSELECT SUBSTRING(randstr,0,8) from datagen_source;
(二)使用Python进行数据分析
import matplotlib.pyplot as plt# 数据准备categories = ['Category A', 'Category B', 'Category C', 'Category D']values = [23, 45, 17, 56]# 创建柱状图plt.figure(figsize=(10, 6)) # 设置图形大小plt.bar(categories, values, color=['blue', 'green', 'red', 'purple']) # 绘制柱状图,可自定义颜色# 添加标题和轴标签plt.title('Sample Bar Chart')plt.xlabel('Categories')plt.ylabel('Values')# 显示数值for i in range(len(values)):plt.text(i, values[i], str(values[i]), ha='center', va='bottom') # 在每个柱子下方显示数值# 显示图形plt.show()
运行后即可获得示例柱状图:
2. 交互式分析
from ipywidgets import interactive,IntSliderquery_age = 20def f(age):global query_agequery_age = ageinteractive(f, age=IntSlider(value=20, min=1, max=100, step=1, description='查询年龄:'))
SELECT '${query_age}' AS age;(三)体验智能助手Copilot
1.SQL改写
,进⼊Copilot Chat功能界⾯,在Chat输⼊框中,输⼊您的改写要求,单击发送,等待Copilot返回结果。
SELECT ds,spu_id,SUM(sales_amt) AS total_sales,COUNT(DISTINCT order_id) AS total_orders,COUNT(DISTINCT sku_id) AS total_skus,COUNT(DISTINCT buyer_id) AS total_buyers,COUNT(DISTINCT buyer_id) / COUNT(DISTINCT order_id) AS avg_buyers_per_order,COUNT(DISTINCT buyer_id) / COUNT(DISTINCT sku_id) AS avg_buyers_per_sku,SUM(sales_amt) / COUNT(DISTINCT order_id) AS avg_sales_per_order,SUM(sales_amt) / COUNT(DISTINCT sku_id) AS avg_sales_per_skuFROM default.dwd_ec_trd_create_ord_diWHERE order_date BETWEEN '2024-09-01' AND '2024-09-18'GROUP BY ds,spu_idORDER BY total_sales DESCLIMIT 10;
修改SQL,将其结果从列转置为行,使用unpivotSELECT season,SUM(tran_amt) AS totalFROM mf_cop_salesPIVOT (SUM(tran_amt) FOR season IN ('Q1' AS spring,'Q2' AS summer,'Q3' AS autumn,'Q4' AS winter))AS pivot_tablel;
解释这段代码CREATE TABLE ods_mbr_user_info(id BIGINT,gmt_create STRING,gmt_modified STRING,id_card_number STRING,id_card_type STRING,is_delete STRING,nick STRING,reg_address STRING,reg_birthdate STRING,reg_city_id STRING,reg_email STRING,reg_fullname STRING,reg_gender STRING,reg_mobile_phone STRING,reg_nation_id STRING,reg_prov_id STRING,user_active_time STRING,user_active_type STRING,user_id BIGINT,user_regdate STRING,user_regip STRING,vip_level STRING)COMMENT '';
为每个字短添加注释