Jonathan Shihao Ji, Ph.D. Director, Intelligent Systems Lab Associate Professor School of Computing, College of Engineering University of Connecticut (UConn)    Office: ITE 361 Email: shihao.ji@uconn.edu

>> Project Pepper >> Project Spot >> UAV3D Benchmark

[Google Scholar, Publications, Teaching, Group, Software, Sponsor, Internal]

Recuriting

Short Bio

I'm an Associate Professor in the School of Computing at University of Connecticut (UConn). My principal research interests lie in the area of deep learning and its applications to computer vision, natural language processing, and robotics, with an emphasis on high-performance computing. I'm interested in developing efficient algorithms that can learn from a variety of data sources (e.g., image, audio, text and time series) on large scale and automate decision-making processes in dynamic environments. I have published over 50 papers in top-ranked journals and prestigious conferences with high impact factors, including CVPR, NeurIPS, ICCV, ECCV, ICLR, ICML, AAAI, AIStats, BMVC, ECML, ICIP, IJCNN, EMNLP, SIGIR, CIKM, COLING, IEEE TPAMI, TSP, TKDE, TPDS, IEEE Internet of Things Journal etc. My research has been supported by federal agencies, including NSF, NIH, DoD, ARO, as well as industry companies, such as VMware, Cisco, Nvidia, and Bill & Melinda Gates Foundation.

I received my Ph.D. in Electrical and Computer Engineering from Duke University, where I worked with Dr. Larry Carin. Before joining UConn, I was an Associate Professor in Computer Science at Georgia State University and Director of the DoD Center of Excellence (CiARE). Prior to returning to academia, I spent approximately 10 years in industry research labs. I am a Senior Member of the IEEE.

1. Yang Li, Shaobo Han, and Shihao Ji, "VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks," The 38th Conference on Neural Information Processing Systems (NeurIPS, IF: 42.3), Vancouver, Canada, Dec. 2024. [arXiv | code]

2. Hui Ye, Raj Sunderraman, and Shihao Ji, "UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles," The 38th Conference on Neural Information Processing Systems (NeurIPS, IF: 42.3), Vancouver, Canada, Dec. 2024. [project website | arXiv]

3. Qing Su, and Shihao Ji, "Unsqueeze [CLS] Bottleneck to Learn Rich Representations," The 18th European Conference on Computer Vision (ECCV, IF: 38.7), Milano, Italy, Sept. 2024. [eccv | arXiv | code]

4. Han Xu, Yutong Li, and Shihao Ji, "LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs," The 10th IEEE World Forum on Internet of Things (WF-IoT), Ottawa, Canada, Nov. 2024. [arXiv]

5. Hui Ye, Raj Sunderraman, and Shihao Ji, "MatchXML: An Efficient Text-label Matching Framework for Extreme Multi-label Text Classification," IEEE Transactions on Knowledge and Data Engineering (TKDE, IF: 8.9), Mar. 2024. [IEEE | arXiv | code]

6. Qing Su, Anton Netchaev, Hai Li, and Shihao Ji, "FLSL: Feature-level Self-supervised Learning," The 37th Conference on Neural Information Processing Systems (NeurIPS, IF: 42.3), New Orleans, Louisiana, USA, Dec. 2023. [neurips | arXiv | code]

7. Xiulong Yang, Qing Su, Shihao Ji, "Towards Bridging the Performance Gaps of Joint Energy-based Models," IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR, IF: 63.1), Vancouver, Canada, June 2023. [paper | code]

8. Yang Li, Xin Ma, Raj Sunderraman, Shihao Ji, Suprateek Kundu, "Accounting for Temporal Variability in Functional Magnetic Resonance Imaging Improves Prediction of Intelligence," Human Brain Mapping (IF: 4.55), July 2023. [paper]

9. Xiulong Yang, and Shihao Ji, "M-EBM: Towards Understanding the Manifolds of Energy-Based Models," The 27th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), Osaka, Japan, May 2023. [paper]

10. Yang Ye, Xiulong Yang, and Shihao Ji, "APSNet: Attention Based Point Cloud Sampling," The 33rd British Machine Vision Conference (BMVC, IF: 5.94), London, UK, Nov. 2022. [paper | code] (Spotlight)

11. Mingchen Li, Shihao Ji, "Semantic Structure based Query Graph Prediction for Question Answering over Knowledge Graph," International Conference on Computational Linguistics (COLING, h5-index: 58), Gyeongju, Republic of Korea, Oct. 2022. [paper | code]

12. Qing Su, Shihao Ji, "Chitransformer: Towards Reliable Stereo From Cues," IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR, IF: 63.1), New Orleans, Louisiana, USA, June 2022. [paper | code]

13. S. Lee, B. Keith, Y. Bey, Y. Wang, X. Yang, X. Li, and Shihao Ji, "A Convenient Rhetoric or Substantial Change of Teacher Racial Diversity? A Text Mining Analysis of Federal, State, and District Documents," Education Policy Analysis Archives (EPAA), Vol. 30, No. 93, June 2022. [paper]

14. Xiulong Yang, Shihao Ji, "JEM++: Improved Techniques for Training JEM," IEEE/CVF International Conference on Computer Vision (ICCV, IF: 40.6), Montreal, Canada, Oct. 2021. [paper | code]

15. Hui Ye, Xiulong Yang, Martin Takac, Raj Sunderraman, and Shihao Ji, "Improving Text-to-Image Synthesis Using Contrastive Learning," The 32nd British Machine Vision Conference (BMVC, IF: 5.94), Virtual, Nov. 2021. [arXiv | code]

16. Xiang Li, Shihao Ji, "Generative Dynamic Patch Attack," The 32nd British Machine Vision Conference (BMVC, IF: 5.94), Virtual, Nov. 2021. [arXiv | code]

17. Yang Li, Shihao Ji, "Dep-L0: Improving L0-based Network Sparsification via Dependency Modeling," The European Conference on Machine Learning (ECML, IF: 4.95), Virtual, Sept. 2021. [arXiv | code]

18. Xiulong Yang, Hui Ye, Yang Ye, Xiang Li, and Shihao Ji, "Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More," The European Conference on Machine Learning (ECML, IF: 4.95), Virtual, Sept. 2021. [arXiv | code]

19. Yang Ye, Shihao Ji, "Sparse Graph Attention Networks," IEEE Transactions on Knowledge and Data Engineering (TKDE, IF: 8.9), Apr. 2021. [IEEE | arXiv | code]

20. Yang Li, Shihao Ji, "Neural Plasticity Networks," International Joint Conference on Neural Networks (IJCNN, IF: 7.09), Virtual, July 2021. [IEEE | arXiv | demo]

21. K. Sarker, X. Yang, Y. Li, S. Belkasim, and Shihao Ji, "A Unified Density-Driven Framework for Effective Data Denoising and Robust Abstention," IEEE International Conference on Image Processing (ICIP, IF: 7.59), Alaska, USA, Sept. 2021. [arXiv]

22. J. Chen, Xiang Li, V. D. Calhoun, J. A. Turner, T. G. M. van Erp, L. Wang, O. A. Andreassen, I. Agartz, L. T. Westlye, E. Jonsson, J. M. Ford, D. H. Mathalon, F. Macciardi, D. S. O'Leary, J. Liu, Shihao Ji, "Sparse Deep Neural Networks on Imaging Genetics for Schizophrenia Case-Control Classification," Human Brain Mapping (IF: 4.55), March 2021. [paper]

23. Xiulong Yang, Shihao Ji, "Learning with Multiplicative Perturbations," 25th International Conference on Pattern Recognition (ICPR, IF: 4.29), Milan, Italy, Jan. 2021. [IEEE | arXiv | code]

24. K. Sarker, S. Pandit, A. Sarker, S. Belkasim, and Shihao Ji, "Reducing Risk and Uncertainty of Deep Neural Networks on Diagnosing COVID-19 Infection," AAAI 2021 Workshop on Trustworthy AI for Healthcare, Virtual, Feb. 2021. [arXiv]

25. K. Sarker, S. Pandit, A. Sarker, S. Belkasim, and Shihao Ji, "Towards Reliable and Trustworthy Computer-Aided Diagnosis Predictions: Diagnosing COVID-19 from X-Ray Images," ACM-CHIL Workshop. [arXiv]

26. Xiang Li, Shihao Ji, "Neural Image Compression and Explanation," IEEE Access (IF: 3.75), Vol. 8, Nov. 30, 2020. [IEEE | arXiv | code]

27. Kaiyang Li, Guangchun Luo, Yang Ye, Wei Li, Shihao Ji, and Zhipeng Cai, "Adversarial Privacy Preserving Graph Embedding against Inference Attack," IEEE Internet of Things Journal (IF: 9.94), Volume 8, Issue 8, Pages 6904-6915, Apr. 2021. [IEEE | arXiv | code]

28. J. Chen, Xiang Li, V. Calhoun, J. Turner, T.G.M. Erp, L. Wang, O. Andreassen, I. Agartz, L. Westlye, J. Liu, and Shihao Ji, "Sparse Deep Neural Networks on Imaging Genetics for Schizophrenia Discrimination," The Organization for Human Brain Mapping (OHBM), Montreal, June 2020.

29. Yang Li, Shihao Ji, "L0-ARM: Network Sparsification via Stochastic Binary Optimization," The European Conference on Machine Learning (ECML, IF: 4.95), Würzburg, Germany, Sept. 2019. [Springer | arXiv | demo | code]

30. Xiang Li, Shihao Ji, "Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks," Machine Learning for Cybersecurity (ECML workshop on MLCS), Würzburg, Germany, Sept. 2019. [Springer | arXiv | code]

31. A. Ahmadzadeh, S. Mahajan, D. Kempton, R. Angryk, and Shihao Ji, "Toward Filament Segmentation Using Deep Neural Networks," 2019 IEEE International Conference on Big Data, Los Angeles, USA, Dec. 2019. [IEEE | arXiv]

32. Shihao Ji, Nadathur Satish, Sheng Li, and Pradeep Dubey, "Parallelizing Word2Vec in Shared and Distributed Memory," IEEE Transactions on Parallel and Distributed Systems (IEEE TPDS), Volume 30, Issue 9, Pages 2090-2100, Sept. 1 2019. [IEEE | arXiv | code]

33. J. Zhang, P. Raman, Shihao Ji, H. Yu, S.V.N. Vishwanathan, I. S. Dhillon, "Extreme Stochastic Variational Inference: Distributed Inference for Large Scale Mixture Models," The 22nd International Conference on Artificial Intelligence and Statistics (AIStats, IF: 9.9), Naha, Okinawa, Japan, Apr., 2019. [pdf]

34. Krishanu Sarker, Mohamed Masoud, Saeid Belkasim, and Shihao Ji, "Towards Robust Human Activity Recognition from RGB Video Stream with Limited Labeled Data," IEEE International Conference on Machine Learning and Applications (ICMLA, acceptance rate 31%), Dec. 2018. [IEEE | arXiv] (SCD'18 Best Poster Award)

35. Shihao Ji, Nadathur Satish, Sheng Li, and Pradeep Dubey, "Parallelizing Word2Vec in Multi-Core and Many-Core Architectures," NIPS workshop on Efficient Methods for Deep Neural Networks, Barcelona, Spain, Dec., 2016. [arXiv | code]

36. Shihao Ji, Hyokun Yun, Pinar Yanardag, Shin Matsushima, and S. V. N. Vishwanathan, "WordRank: Learning Word Embeddings via Robust Ranking," Conference on Empirical Methods in Natural Language Processing (EMNLP), Nov., 2016. [arXiv | code]

37. Shihao Ji, S. V. N. Vishwanathan, Nadathur Satish, Michael J. Anderson, and Pradeep Dubey, "Blackout: Speeding up recurrent neural network language models with very large vocabularies," International Conference on Learning Representations (ICLR Oral, acceptance rate 6%), May, 2016. [arXiv | code]

38. O. Chapelle, Shihao Ji, C. Liao, E. Velipasaoglu, L. Lai, and S.-L. Wu, "Intent-based diversification of web search results: Metrics and algorithms," Information Retrieval Journal, May, 2011. [pdf file]

39. Shihao Ji, K. Zhou, C. Liao, Z. Zheng, G.-R. Xue, O. Chapelle, G. Sun, and H. Zha, "Global ranking by exploiting user clicks," In SIGIR '09: Proceedings of the 32nd Annual International ACM SIGIR conference on Research and development in information retrieval, July, 2009. [pdf file]

40. T. Moon, Shihao Ji, G. Dupret, C. Liao, and Z. Zheng, "User behavior driven ranking without editorial judgment," The 19th ACM International Conference on Information and Knowledge Management (CIKM), Oct. 2010. [pdf file]

41. X. Li, F. Li, Shihao Ji, Z. Zhaohui, Y. Chang, and A. Dong, "Incorporating robustness into web ranking function selection," The 18th ACM International Conference on Information and Knowledge Management (CIKM), Nov. 2009. [paper]

42. A. Dong, Y. Chang, Shihao Ji, C. Liao, X. Li, and Z. Zheng, "Empirical exploitation of click data for topical ranking," Conference on Empirical Methods in Natural Language Processing (EMNLP), Aug. 2009. [paper]

43. F. Li, X. Li, Shihao Ji, and Z. Zheng, "Comparing both relevance and robustness in selection of web ranking functions," In SIGIR '09: Proceedings of the 32nd Annual International ACM SIGIR Conference, July 2009. [paper]

44. Shihao Ji, D. Dunson, and L. Carin, "Multi-task compressive sensing," IEEE Trans. Signal Processing, vol. 57, no. 1, pp. 92-106, Jan. 2009. [pdf file | code]

45. Shihao Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Processing, vol. 56, no. 6, pp. 2346-2356, June 2008. [pdf file | code]

46. T. Wang, T. Furey, J. Connelly, Shihao Ji, S. Nelson, S. Heber, S. Gregory, and E. Hauser, "A general integrative genomic feature transcription factor binding site prediction method applied to analysis of USF1 binding in cardiovascular disease," Human Genomics, vol. 3, no. 3, pp. 221-235, Apr. 2009. [paper]

47. Shihao Ji, L. T. Watson, and L. Carin, "Semi-supervised learning of hidden Markov models via a homotopy method," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 275-287, Feb. 2009. [pdf file | data | demo]

48. J. Fang, Shihao Ji, Y. Xue, and L.Carin, "Multi-task classification by learning the task relevance," IEEE Signal Processing Letters, vol. 15, pp. 593-596, 2008. [paper]

49. Shihao Ji, R. Parr, H. Li, X. Liao, and L. Carin, "Point-based policy iteration," In the 22nd National Conference on Artificial Intelligence (AAAI), Vancouver, Canada, July 22-26, 2007. [pdf file]

50. Shihao Ji and L. Carin, "Bayesian compressive sensing and projection optimization," In the 24th International Conference on Machine Learning (ICML), Corvallis, Oregon, June 20-24, 2007. [pdf file]

51. Shihao Ji and L. Carin, "Cost-sensitive feature acquisition and classification," Pattern Recognition, vol. 40, no. 5, pp. 1474-1485, May 2007. [pdf file]

52. Shihao Ji, R. Parr, and L. Carin, "Non-myopic multi-aspect sensing with partially observable Markov decision processes," IEEE Trans. Signal Processing, vol. 55, no. 6, pp. 2720-2730 June 2007. [pdf file | demo1 ,demo2 ,demo3]

53. L. He, Shihao Ji, W.R. Scott, and L. Carin, "Adaptive multi-modality sensing of landmines," IEEE Trans. Geoscience and Remote Sensing, vol. 45, no. 6, pp.1756-1774, June 2007. [pdf file]

54. Shihao Ji, B. Krishnapuram, and L. Carin, "Variational Bayes for continuous hidden Markov models and its application to active learning," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 522-532, Apr. 2006. [pdf file | data]

55. N. Dasgupta, Shihao Ji, and L. Carin, "Homotopy-based semi-supervised hidden Markov tree for texture analysis," in Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2006. [pdf file]

56. L. He, Shihao Ji, and L. Carin, "Application of partially observable Markov decision processes to robot navigation in a minefield," in ICAPS Workshop on POMDP, Classification and Regression: Relationships and Joint Utilization, June 2006. [pdf file]

57. Shihao Ji, X. Liao, and L. Carin, "Adaptive multi-aspect target classification and detection with hidden Markov models," IEEE Sensors Journal, vol. 5, no. 5, pp. 1035-1042, Oct. 2005. [pdf file]

58. Shihao Ji, X. Liao, and L. Carin, "Adaptive multi-aspect target classification and detection with hidden Markov models," in Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol. 2, pp. 125-128, May 2004. [pdf file]

• CSC 8851 - Deep Learning (Fall 2023)
• CSC 8980 - Deep Reinforcement Learning (Spring 2023)
• CSC 8851 - Deep Learning (Spring 2022)
• CSC 8851 - Deep Learning (Fall 2021)
• CSC 4851/6851 - Intro to Deep Learning (Spring 2021)
• CSC 8851 - Deep Learning (Fall 2020)
• CSC 4980/6980 - Intro to Deep Learning (Spring 2020)
• CSC 8851 - Deep Learning (Fall 2019)
• CSC 4980/6980 - Intro to Deep Learning (Spring 2019)
• CSC 8980 - Deep Learning (Fall 2018)
• CSC 8980 - Deep Learning (Spring 2018)
• CSC 4740/6740 - Data Mining (Fall 2017)

• VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks
• UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles
• SADA-JEM: Towards Bridging the Performance Gaps of Joint Energy-based Models
• ChiTransformer: Chitransformer: Towards Reliable Stereo From Cues
• T2I-CL: Improving Text-to-Image Synthesis Using Contrastive Learning
• JEM++: Improved Techniques for Training JEM
• GDPA: Generative Dynamic Patch Attack
• Dep-L0: Improving L0-based Network Sparsification via Dependency Modeling
• GMMC: Generative Max-Mahalanobis Classifiers for Image Classification, Generation and More
• SGAT: Sparse Graph Attention Networks
• xVAT: Learning with Multiplicative Perturbations
• L0-arm: Network Sparsification via Stochastic Binary Optimization
• Defense-VAE: A Fast and Accurate Defense against Adversarial Attacks
• Parallel Word2Vec: Parallelizing Word2Vec in Shared and Distributed Memory
• BlackOut: Speeding up RNNLMs with very large vocabularies
• WordRank: Learning Word Embeddings via Robust Ranking
• BCS: a Bayesian framework for solving the inverse problem of compressive sensing

1. Visualization of part of the neurons in (a) conv-layer and (b) fully-connected layer of the LeNet-5-Caffe sparsified by L0-ARM.
2. Visualization of Neural Plasticity Networks on sythetic "moons" dataset for (a) network sparsification and (b) network expansion.
3. Visualization of SGAT on the graph of Zachary’s Karate Club to prune task-irrelevant edges.
4. Project Pepper
5. Project Spot