爬虫

什么是爬虫?

顾名思义:会爬的虫子🐛(bushi

爬虫其背后的意义其实非常广泛,简单来讲,它们是一段计算机程序,按照我们定义的规则,在互联网上提取相应的信息或完成一定的操作,我们的生活中无处不充满爬虫。比如 飞机、火车抢票,全中国火车票票源只有12306,那么其他售票软件的票是哪来的呢,答案是他们每天放出无数爬虫对12306的售票接口进行查询,一旦有票就自动抢票,这样就是为什么你总是抢不到票(doge)。意思大概就是这么个意思,懂得都懂,不懂我也没办法,关于这个我只能说这么多。

怎么写一个爬虫?

爬虫其实按步骤分只有以下简单的几步:

  1. 确定目标网站,并调用相应的模块发起http(s)请求
  2. 使用相应的模块来解析html文档
  3. 利用程序分析、提取文档中的信息

一个小例子

职教云为例,要爬取资源库里的专业课信息,我们首先进入某一个专业

课程中心

进入开发工具查看请求

查看网络请求

可以看到网站是通过ajax发送请求获取数据的,所以我们以python为例,可以这么写:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import requests

headers = {
'Accept': 'application/json, text/javascript, */*; q=0.01',
'Referer': 'https://www.icve.com.cn/portalproject/themes/default/vw7vanamblnap617thvjxa/sta_page/course_center.html?projectId=vw7vanamblnap617thvjxa',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.1 Safari/605.1.15',
'X-Requested-With': 'XMLHttpRequest',
}

params = (
('projectId', 'vw7vanamblnap617thvjxa'),
('hot', 'false'),
('pagesize', '200'),
('page', '1'),
('order', 'asc'),
)

def getCourseInfo(): # 获取html
response = requests.get(
'https://www.icve.com.cn/portalProject/ajax-portalpoject-course', headers=headers, params=params)
return response.text


if __name__ == "__main__":
html = getCourseInfo()
print(html)

这样我们就拿到了这段json数据:image-20211111163249892

怎么处理?

通过观察json格式,可以发现,所需数据在/list/obj下,所以编写代码取得list下的数据

1
2
3
4
def getData(html):
df = json.loads(html) # 调用json.loads()方法将html文档转换成dict
data = df['list'][7:-1] # 选取dict中的list,并进一步提取json
return data

如此,我们便获得了一份干净的json数据

1
[{"id":"vgxoanmnaz9g9sqkmox2fw","coursetitle":"计算机应用基础","DateCreated":"2017-09-17T15:07:26","CourseDes":"熟练掌握Windows操作系统的基本知识、熟练操作Office办公软件是现如今每个人都要具备的最基本的技能。本课程讲解通俗易懂,深入浅出,适合零基础的人来学习,还犹豫什么?赶紧来吧!","cover":"https://file.icve.com.cn/doc_public4/233/812/3A4CB2EE2DF850CF82A59EDC71AD7109.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"9chkajuoap9p820mb1-laq","peoplecount":18082,"tags":"计算机应用","displayname":"眭碧霞","CourseCategory":1,"courseType":"示范课","PrintingStates":1,"unitname":"常州信息职业技术学院","time":"2017.09.17"},{"id":"9r8fafqmbadnxjoeiw-qhg","coursetitle":"C语言程序设计","DateCreated":"2017-09-22T16:39:47","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/112/603/1C396C5F363CFEA7132E9A4A08568820.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"yrhzan2mblnmnjwr2-tsfq","peoplecount":11471,"tags":"c语言","displayname":"李学刚","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"常州信息职业技术学院","time":"2017.09.22"},{"id":"ejv-ao-nvzzngaccwjtypq","coursetitle":"Python程序设计","DateCreated":"2017-10-31T10:44:40","CourseDes":"《Python程序设计》是一门Python的入门基础课程,面向各个专业方向零编程基础的所有学员。如果您有兴趣从零开始学习Python程序设计,选择这门课程是您最佳选择!我们将带您进入Python的世界!","cover":"https://file.icve.com.cn/file_doc_public/357/1011/597FCED629D6F49D8FA97DC299F5FB0B.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"wturaeso-yppni8frhagwq","peoplecount":8143,"tags":"python","displayname":"黄锐军","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"深圳信息职业技术学院","time":"2017.10.31"},{"id":"1pl7aeeomodafo4j1oh3yw","coursetitle":"网页设计与制作","DateCreated":"2017-12-25T10:43:29","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/309/969/4D6F2606FCAA24D56392FD286A9EF600.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"bhhgamylyaflwpwy-uihqg","peoplecount":5866,"tags":"网页设计,网页制作,html5,css3,JavaScript","displayname":"章早立","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"江苏电子信息职业学院","time":"2017.12.25"},{"id":"vbjaaoqq34bkrillc8z3vg","coursetitle":"Java程序设计","DateCreated":"2019-10-18T16:47:26","CourseDes":"本课程是培养Java平台开发工程师、Android手机开发工程师的必修课程。通过本课程的学习,可使学生掌握使用Java语言设计和开发面向对象应用程序的知识和技能,具备Java开发工程师和Android开发工程师的基本职业能力。","cover":"https://file.icve.com.cn/file_doc_public/810/149/CAA254116C46B69D4DC33C368FFBEEB2.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"0qzhaksoeahabsypodfzkq","peoplecount":5213,"tags":"Java","displayname":"杨丹","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2019.10.18"},{"id":"59ynaf6pjo1ajtx227s31q","coursetitle":"JavaScript及框架应用","DateCreated":"2018-09-20T13:51:46","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/395/384/62E600EA00DE0884DCC4B22FC34D3F7C.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"dmuxawsoy5japinjtu0eia","peoplecount":3087,"tags":"JavaScript","displayname":"郑丽萍","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"江苏电子信息职业学院","time":"2018.09.20"},{"id":"3f-uaeiogr9gd9ubguqeyq","coursetitle":"软件测试","DateCreated":"2017-12-20T14:34:40","CourseDes":"软件测试","cover":"https://file.icve.com.cn/doc_public4/669/652/A77A32C046828D02A8F96A32BC70E7A1.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"nrlladknyq5frpzyo1rvzg","peoplecount":2962,"tags":"软件测试","displayname":"郭雷","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"南京工业职业技术大学","time":"2017.12.20"},{"id":"5shxadko0yxc4c9uhijwvg","coursetitle":"C#程序设计","DateCreated":"2017-12-10T11:43:20","CourseDes":"通过本课的学习,使学生掌握C#语言的语法知识,理解和掌握面向对象程序设计的思想和方法,能熟练使用Visual Studio集成开发环境编写、调试和测试控制台应用软件、Winform应用软件;运用ADO.NET开发数据库应用程序,具有面向对象程序设计、分析和调试能力。","cover":"https://file.icve.com.cn/doc_public4/489/815/7A7CBC695D749E8174BD3B49E9AED35E.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"jm5caukok71fo14fmnz5vq","peoplecount":2548,"tags":"C#,程序设计","displayname":"郑卉","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"重庆电工程职业学院","time":"2017.12.10"},{"id":"3pchafqrtldictqruhsl1q","coursetitle":"计算机应用(Win10+Office2016)","DateCreated":"2020-07-16T17:16:45","CourseDes":"熟练掌握Windows操作系统的基本知识、熟练操作Office办公软件是现如今每个人都要具备的最基本的技能。本课程讲解通俗易懂,深入浅出,适合零基础的人来学习,还犹豫什么?赶紧来吧!","cover":"https://file.icve.com.cn/doc_public4/542/578/87990BF45A2B5A62AB6BE63944985602.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"9chkajuoap9p820mb1-laq","peoplecount":2334,"tags":"计算机应用,Windows10,Office2016","displayname":"眭碧霞","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2020.07.16"},{"id":"bsdab6qlylet2arffqjpw","coursetitle":"信息技术基础","DateCreated":"2019-10-15T10:31:43","CourseDes":"作为新时代的社会人,怎么能不了解云计算、大数据、物联网、移动互联和人工智能等新技术呢?咱们的课程将带你走入新兴信息技术的天地,零基础也不怕!选择我们的课程,不会让你失望!","cover":"https://file.icve.com.cn/file_doc_public/1/697/005AE696CDA534CA0A884F42314181CA.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"9chkajuoap9p820mb1-laq","peoplecount":2212,"tags":"信息技术,云计算,大数据,人工智能,物联网,移动互联","displayname":"眭碧霞","CourseCategory":1,"courseType":"示范课","PrintingStates":1,"unitname":"常州信息职业技术学院","time":"2019.10.15"},{"id":"8wgtaaooharpxsgvxpqelg","coursetitle":"数据结构","DateCreated":"2017-12-12T17:07:01","CourseDes":"微课视频、PPT课件、动画、线上线下混合教学、翻转课堂、资源库","cover":"https://file.icve.com.cn/doc_public4/814/955/CB8EEE15EB392A26E8C677F7E8DD7358.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"ssmwalyn06pnjcupjwha3q","peoplecount":2125,"tags":"数据结构,算法设计,数据存储","displayname":"李刚","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"江苏电子信息职业学院","time":"2017.12.12"},{"id":"3fdtaaeogpndjac1zhi-9w","coursetitle":"HTML5程序设计","DateCreated":"2017-12-20T19:23:08","CourseDes":"《HTML5程序设计》是软件技术及相关专业的一门以HTML5和CSS3基本标记为基础,同时拓展介绍HTML5核心技术的一门专业课程,可以面向对HTML5感兴趣的所有学员,无论你有没有基础,通过这门课程的资源库,你都能找到你所需要的,一起来学习吧!","cover":"https://file.icve.com.cn/doc_public4/662/25/A5A06435B51B3C58DAD194838593C030.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"ujhoaeko-b5eszhmq1do-w","peoplecount":2106,"tags":"HTML5","displayname":"任文娟","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"山东电子职业技术学院","time":"2017.12.20"},{"id":"8ggbawiqnondttuqqkhetw","coursetitle":"Java程序设计","DateCreated":"2019-06-12T11:08:29","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/446/928/6FAE82EA00357B8A62CB1FBBC3399DBB.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"li4davwqrbxgmukoxjdqw","peoplecount":2011,"tags":"Java程序设计","displayname":"徐红","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"山东商业职业技术学院","time":"2019.06.12"},{"id":"chioaemozktkntlbgtnbca","coursetitle":"移动软件UI设计","DateCreated":"2017-12-14T14:22:11","CourseDes":"本课程是软件技术专业的一门实践性很强的专业提升课。培养学生对软件的人机交互、操作逻辑、界面美观的整体设计。好的UI设计不仅是让软件变得有个性有品位,还要让软件的操作变得舒适简单、自由,充分体现软件的定位和特点。","cover":"https://file.icve.com.cn/doc_public4/637/718/9F4B39D80E0F92C3C068174BD9FBDA1C.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"rqlyaukozbxjdpmdim9scw","peoplecount":1455,"tags":"UI,UI界面设计","displayname":"沈涵","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"苏州信息职业技术学院","time":"2017.12.14"},{"id":"pzqgaeso45jncpwnmlujha","coursetitle":"JAVA程序设计","DateCreated":"2017-12-20T19:24:11","CourseDes":"Java是一种可以撰写跨平台应用软件的面向对象的程序设计语言,现已成为当前计算机软件主流开发语言之一,为众多的程序员所使用。Java 技术广泛应用于个人PC、数据中心、游戏控制台、科学超级计算机、移动电话和互联网,同时拥有全球最大的开发者专业社群。让我们一起遨游其中吧!","cover":"https://file.icve.com.cn/doc4/112/132/1C1210C9A25F68DC825C20284BF927C4.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"ft-aalwncaraqoej9n2iya","peoplecount":1412,"tags":"JAVA","displayname":"王轶凤","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"山东商业职业技术学院","time":"2017.12.20"},{"id":"bpm5af2oca1cbzcgdhglsg","coursetitle":"ASP.NET程序设计","DateCreated":"2018-01-04T16:22:52","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/978/641/F4BA043D4CDAAB87CEC921F438B8443F.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"oigzaesooi5ettz-gua9ra","peoplecount":1198,"tags":"asp.net,网站开发","displayname":"徐占鹏","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"青岛职业技术学院","time":"2018.01.04"},{"id":"dgsxaaqokpvoyhp18r-etg","coursetitle":"软件开发与项目管理","DateCreated":"2017-11-30T10:57:11","CourseDes":"是软件技术专业课程体系中一门综合性、实践性很强的专业课程。通过本课程学习,学生应了解软件项目需求、设计、编码、测试、维护和管理的一般过程,学会软件开发的方法,能规范撰写软件开发过程基本技术文档,具有熟练的软件编码能力,掌握软件测试的基本方法,为今后从事软件工程实践打下良好的基础。","cover":"https://file.icve.com.cn/doc_public4/869/200/D943209D9DA273EF16A5329F9CDA0443.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"ow8dab2n9axgm0sivsugbq","peoplecount":1157,"tags":"软件开发,项目管理,综合项目开发","displayname":"朱利华","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"常州信息职业技术学院","time":"2017.11.30"},{"id":"7dhfagsqjp1arfag94ytra","coursetitle":"PHP动态网站开发实例教程","DateCreated":"2019-06-10T11:17:11","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/440/366/6E15BA1D826E43C195B22FBE0630774F.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"lsk0ag6ndireqfbi7sqw5a","peoplecount":1117,"tags":"PHP动态网站开发","displayname":"钱兆楼","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"江苏电子信息职业学院","time":"2019.06.10"},{"id":"9lsgaesojrpp98e2zzs-eg","coursetitle":"PHP程序设计","DateCreated":"2018-01-04T16:40:58","CourseDes":"PHP是一种创建动态交互性网站的强有力的服务器端脚本语言,开源、免费,使用非常广泛,同时也是网络程序设计、网站开发需要掌握的重要语言。通过本课程的学习,可以掌握PHP语言的基本语法、高级应用和数据库编程等;掌握PHP程序设计的基本思想和开发过程,培养良好的编程规范和职业习惯。","cover":"https://file.icve.com.cn/doc_public4/525/313/8354E6DDCFA5F4D94B7BFE367EB15BBA.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"hkmqabqneoriotvwlyqsgq","peoplecount":1082,"tags":"PHP,PHP程序设计,动态网页,网站开发,PHP教程,php语言,PHP语法,PHP基础,PHP数据库编程","displayname":"张光辉","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"四川信息职业技术学院","time":"2018.01.04"},{"id":"4khagsqobbjg82u8ygra","coursetitle":"软件工程与UML","DateCreated":"2019-06-10T11:17:39","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/727/890/B5FDE9F1D445E18B4DEDB6DF3AF8FA03.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"z8spac-pc5pmsujgnzvdpa","peoplecount":1055,"tags":"软件工程","displayname":"罗炜","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"武汉软件工程职业学院","time":"2019.06.10"},{"id":"m5heakutoofgcmlbf5nsyw","coursetitle":"Python程序设计基础","DateCreated":"2021-09-24T13:47:59","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/190/541/2F98747C35B48ECC820279EC68E5BCBB.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"e1tagirukln14mdr34spg","peoplecount":1050,"tags":"python","displayname":"虞菊花","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.09.24"},{"id":"ylpabqs6zfggxeqg3mxq","coursetitle":"数据结构(Java语言描述)","DateCreated":"2020-08-19T08:56:59","CourseDes":"要想编写出一个优良的程序,就必须处理好对象的特征和各个对象之间的关系,这就是数据结构课程所要研究的重点,本课程将带领你研究数据在计算机中的存储和处理方法,培养分析数据、组织数据、应用结构优化的数据进行高效编程的能力,培养数据抽象能力和复杂程序设计能力!","cover":"https://file.icve.com.cn/file_doc_public/11/164/02D292F17804096EBC821D34E0E45FDB.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"zwreapundzfin5zducndtg","peoplecount":991,"tags":"数据结构,Java","displayname":"张静","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2020.08.19"},{"id":"56wfapmno55pywt1lvezag","coursetitle":"数据库管理与应用","DateCreated":"2017-12-07T09:50:48","CourseDes":"数据库技术是目前IT行业中发展最快的领域之一,已经被广泛应用于各种类型的数据处理系统之中。了解并掌握数据库知识已经成为各类科技人员和管理人员的基本要求。","cover":"https://file.icve.com.cn/doc_public4/423/449/69C705839B7ACCD70A21E43BFBEE6A73.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"rqlyaukozbxjdpmdim9scw","peoplecount":984,"tags":"SQL SERVER","displayname":"沈涵","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"苏州信息职业技术学院","time":"2017.12.07"},{"id":"e6chadkosklkl0knsusj0a","coursetitle":"JAVA Web程序设计","DateCreated":"2018-01-28T15:02:15","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/612/124/9901F27CD30975C6F342AC90B450D8C5.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"n65-alwnoqrips9i0sencg","peoplecount":703,"tags":"jsp,web开发","displayname":"宁云智","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"湖南铁道职业技术学院","time":"2018.01.28"},{"id":"84unalgozbtlpfqdxwlkuw","coursetitle":"JavaEE企业级项目开发","DateCreated":"2018-03-28T09:45:45","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/503/636/7DF9F0E3E0661501646DA98E3904D9B8.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"8scaab2ntkjm38sp1izf6w","peoplecount":698,"tags":"JavaEE,SSH","displayname":"蒋卫祥","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2018.03.28"},{"id":"cmgcasaopjjjyq4qp682bg","coursetitle":"移动软件开发","DateCreated":"2018-01-20T12:45:54","CourseDes":"课程主要是学习了使用swift语言进行iOS移动应用软件开发,完整了解iOS移动应用开发的整个过程,并且培养自己独立开发一款常规iOS移动应用的能力。","cover":"https://file.icve.com.cn/doc_public4/234/701/3A9AF69BEE1F7081CBBA3016B8ACBB20.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"icz3acqoqihkmxtdegon8g","peoplecount":623,"tags":"iOS Swift","displayname":"陈志峰","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"苏州经贸职业技术学院","time":"2018.01.20"},{"id":"kxybau6orp5fhjavifpk8w","coursetitle":"JavaEE开发案例(图书馆管理系统的设","DateCreated":"2017-12-20T16:23:28","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/643/872/A0FDA3B5A2AA77FCCDD9D2605CD0E30A.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"yysrabqnxyxmpz41mwd5fq","peoplecount":522,"tags":"JavaEE,系统开发","displayname":"顾晓燕","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"江苏信息职业技术学院","time":"2017.12.20"},{"id":"dehyaruogpxjcrixv4slpw","coursetitle":"Android课程案例","DateCreated":"2017-12-27T08:19:42","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/959/464/EFD74056A2D85BA4F6FE5A38D5F219FD.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"o65-alwnp4xbr3c1hpgrjq","peoplecount":514,"tags":"案例,Android","displayname":"张成年","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"无锡商业职业技术学院","time":"2017.12.27"},{"id":"je9waytap5dajrzwtlnvg","coursetitle":"信息技术基础(第2版)","DateCreated":"2021-08-26T21:51:11","CourseDes":"作为新时代的社会人,怎么能不了解云计算、大数据、物联网、移动互联和人工智能等新技术呢?咱们的课程对接教育部信息技术新课标,将带你走入新兴信息技术的天地,零基础也不怕!选择我们的课程,不会让你失望!","cover":"https://file.icve.com.cn/file_doc_public/924/256/E7240334B39ED056640D1315FE8A1127.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"zwreapundzfin5zducndtg","peoplecount":508,"tags":"信息技术","displayname":"张静","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.08.26"},{"id":"ztpmateonldkrymlkerdkq","coursetitle":"移动App案例设计","DateCreated":"2017-12-07T20:39:02","CourseDes":"本案例包含用户登录模块、书籍查询模块、书籍详细模块、书籍购买模块等多个功能模块;本案例包含了Android学习中的部分技术知识点,除了界面UI和四大组件之外,还包括数据存储、HTTP网络编程、Handler及多线程、JSON解析和封装等。","cover":"https://file.icve.com.cn/doc_public4/198/328/31A522DC793F65204011464AA003E14D.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"jm5caukok71fo14fmnz5vq","peoplecount":476,"tags":"移动书城管理系统","displayname":"郑卉","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"重庆电工程职业学院","time":"2017.12.07"},{"id":"s72haeso2pzi6lkf1r6ruw","coursetitle":"心随行动旅游系统","DateCreated":"2017-12-26T10:46:11","CourseDes":"","cover":"https://file.icve.com.cn/doc_public4/38/127/0991FC1C813D5ED19E9B396E394D2C19.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"ft-aalwn-kxn-phs-dgzvq","peoplecount":453,"tags":"旅游","displayname":"李敬文","CourseCategory":1,"courseType":"示范课","PrintingStates":null,"unitname":"淄博职业学院","time":"2017.12.26"},{"id":"mwewanysrxkhkxgkaumww","coursetitle":"Java Web应用开发","DateCreated":"2021-06-01T16:36:04","CourseDes":"Java Web应用开发是软件技术相关专业必修的专业核心课程,我们将通过详实的内容和丰富的案例,一起学习JSP、JavaBean、Servlet等技术,深入理解MVC模式的设计思想,最终使您能独立开发简单的Web应用程序。快来加入我们的课程,一起开启学习之旅吧!","cover":"https://file.icve.com.cn/file_doc_public/445/480/6F4780E0514F87102C1FC650A7F9AC52.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"ow8dab2n9axgm0sivsugbq","peoplecount":282,"tags":"JSP,Servlet,JavaBean,MVC","displayname":"朱利华","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.06.01"},{"id":"iq6gakmrjrbjeehc9vf3vg","coursetitle":"人工智能导论","DateCreated":"2021-04-08T10:19:12","CourseDes":"课程内容涵盖人工智能技术、运作平台、行业应用、产品和服务以及安全与伦理。人工智能技术通过项目驱动方式讲解,学员一边学习一边编码实现算法,掌握技术和技能。行业应用部分可以采用讨论课形式,安排学员先看微课视频,选择感兴趣的应用收集整理资料,课堂由学生自主讲解,教师安排讨论和点评。","cover":"https://file.icve.com.cn/file_doc_public/710/399/B1B63C8A5566D9C668246675F972014E.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"fnasadkobkfersoranjhlq","peoplecount":280,"tags":"人工智能","displayname":"钱银中","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.04.08"},{"id":"1oguadmsdlvcojhire8x5w","coursetitle":"JavaEE企业级项目开发","DateCreated":"2021-03-15T14:35:46","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/613/546/9948899C25C3EC21A0AC93FEFEEF7485.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"8scaab2ntkjm38sp1izf6w","peoplecount":273,"tags":"Spring,Spring MVC,MyBatis","displayname":"蒋卫祥","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.03.15"},{"id":"5pzfalasxzvghzfromg9q","coursetitle":"Static Web Design Technology","DateCreated":"2021-01-26T18:29:34","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/15/1008/03FFC361671CC7410F3D998D0AB8A6FF.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"theuadgpfynciyhkfpil-q","peoplecount":257,"tags":"HTML,CSS","displayname":"唐小燕","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.01.26"},{"id":"jh4afotnjrgyokftwwtg","coursetitle":"PHP程序设计","DateCreated":"2021-06-27T14:08:16","CourseDes":"本课程主要讲授PHP语言编程基础及LARAVEL框架设计技术,培养学生具有动态网页设计开发能力。引导学生掌握使用LARAVEL框架开发中小型Web应用系统的相关知识与技能。\n在课程的学习中,培养诚实、守信、坚韧不拔的性格,培养善于沟通表达、善于自我学习、具备团队协作的能力。","cover":"https://file.icve.com.cn/file_doc_public/355/396/58C6312BD09AA98E524A57B8E97464EB.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"yv34akmoxjln8-cjhr6uuq","peoplecount":239,"tags":"WEB前端,PHP,Laravel,Mysql开发","displayname":"张金姬","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.06.27"},{"id":"btonaokseobibrqj7lmv6q","coursetitle":"Linux基础","DateCreated":"2021-08-27T17:55:20","CourseDes":"Linux 是目前最流行的服务器操作系统,继承了 Unix 系统的强大功能和极高的稳定性,可以毫不夸张的说,Linux 已经成为人工智能、物联网、5G、大数据和云计算等新一代信息技术的重要基础设施,学习 Linux 操作系统是开启通向未来 IT 技术大门的钥匙。","cover":"https://file.icve.com.cn/file_doc_public/800/610/C8098A1974F85173B812D2D1BB36CEA1.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"hvgpaeqou75avgdgd4t9xw","peoplecount":189,"tags":"Linux,CentOS7","displayname":"胡丽英","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.08.27"},{"id":"vcubaaqtmj9eqgzefariba","coursetitle":"容器技术","DateCreated":"2021-04-14T21:58:41","CourseDes":"","cover":"https://file.icve.com.cn/file_doc_public/270/512/43980324FC7A1D61FC456C977416C2B0.jpg?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"kvqoasuqn5pnjk2rg4kw","peoplecount":172,"tags":"Docker,容器,容器技术,Dockerfile","displayname":"朱川","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.04.14"},{"id":"ondzayqturvkcspzk3xvsa","coursetitle":"Node.js应用开发","DateCreated":"2021-09-16T10:34:35","CourseDes":"Node.js是一个基于 Chrome V8 引擎的 JavaScript 运行环境,因其使用事件驱动的非阻塞 I/O 模型而得以轻量高效,现已成为实时Web应用开发的独特选择。Node.js发展迅速,目前已成为JavaScript服务器端运行平台的事实标准。","cover":"https://file.icve.com.cn/file_doc_public/399/559/63D8BDEAC40E94F7E6B1F46B9B8B54AD.png?x-oss-process=image/resize,m_fixed,w_220,h_165,limit_0","uid":"theuadgpfynciyhkfpil-q","peoplecount":73,"tags":"Node.js,Express,Koa,数据库开发,应用开发","displayname":"唐小燕","CourseCategory":1,"courseType":"示范课","PrintingStates":0,"unitname":"常州信息职业技术学院","time":"2021.09.16"}]

导出

这里用到的是python的pandas模块,使用方法就不赘述了,可以通过百度自行了解其用法

1
2
3
def outPutFile(data):
df = pd.read_json(StringIO(data)) # 这里用到了StringIO模块来避免字符串转义所出现的问题
df.to_csv(r'output.csv', encoding="utf_8_sig")

导出来的结果就是这个样子

image-20211112093608100